Class cascading.detail.EveryPipeAssemblyPlatformTest

72

tests

0

failures

0

ignored

36.962s

duration

100%

successful

Tests

Test Duration Result
-1_'field'_'field'[hadoop2-tez] 1.092s passed
-1_'field'_-1[hadoop2-tez] 4.014s passed
-1_'field'_ALL[hadoop2-tez] 0.820s passed
-1_'field'_REPLACE[hadoop2-tez] 0.075s passed
-1_'field'_RESULTS[hadoop2-tez] 0.841s passed
-1_'field'_SWAP[hadoop2-tez] 0.767s passed
-1_ARGS_'field'[hadoop2-tez] 0.052s passed
-1_ARGS_-1[hadoop2-tez] 0.602s passed
-1_ARGS_ALL[hadoop2-tez] 0.049s passed
-1_ARGS_REPLACE[hadoop2-tez] 0.570s passed
-1_ARGS_RESULTS[hadoop2-tez] 0.653s passed
-1_ARGS_SWAP[hadoop2-tez] 0.567s passed
-1_UNKNOWN_'field'[hadoop2-tez] 0.071s passed
-1_UNKNOWN_-1[hadoop2-tez] 0.720s passed
-1_UNKNOWN_ALL[hadoop2-tez] 0.674s passed
-1_UNKNOWN_REPLACE[hadoop2-tez] 0.566s passed
-1_UNKNOWN_RESULTS[hadoop2-tez] 0.627s passed
-1_UNKNOWN_SWAP[hadoop2-tez] 0.720s passed
-1_VALUES_'field'[hadoop2-tez] 0.055s passed
-1_VALUES_-1[hadoop2-tez] 0.782s passed
-1_VALUES_ALL[hadoop2-tez] 0.717s passed
-1_VALUES_REPLACE[hadoop2-tez] 0.625s passed
-1_VALUES_RESULTS[hadoop2-tez] 0.727s passed
-1_VALUES_SWAP[hadoop2-tez] 0.574s passed
0_'field'_'field'[hadoop2-tez] 0.594s passed
0_'field'_-1[hadoop2-tez] 0.575s passed
0_'field'_ALL[hadoop2-tez] 0.568s passed
0_'field'_REPLACE[hadoop2-tez] 0.050s passed
0_'field'_RESULTS[hadoop2-tez] 0.549s passed
0_'field'_SWAP[hadoop2-tez] 0.569s passed
0_ARGS_'field'[hadoop2-tez] 0.051s passed
0_ARGS_-1[hadoop2-tez] 0.520s passed
0_ARGS_ALL[hadoop2-tez] 0.048s passed
0_ARGS_REPLACE[hadoop2-tez] 0.624s passed
0_ARGS_RESULTS[hadoop2-tez] 0.538s passed
0_ARGS_SWAP[hadoop2-tez] 0.610s passed
0_UNKNOWN_'field'[hadoop2-tez] 0.047s passed
0_UNKNOWN_-1[hadoop2-tez] 0.547s passed
0_UNKNOWN_ALL[hadoop2-tez] 0.559s passed
0_UNKNOWN_REPLACE[hadoop2-tez] 0.472s passed
0_UNKNOWN_RESULTS[hadoop2-tez] 0.551s passed
0_UNKNOWN_SWAP[hadoop2-tez] 0.547s passed
0_VALUES_'field'[hadoop2-tez] 0.048s passed
0_VALUES_-1[hadoop2-tez] 0.471s passed
0_VALUES_ALL[hadoop2-tez] 0.554s passed
0_VALUES_REPLACE[hadoop2-tez] 0.448s passed
0_VALUES_RESULTS[hadoop2-tez] 0.454s passed
0_VALUES_SWAP[hadoop2-tez] 0.472s passed
ALL_'field'_'field'[hadoop2-tez] 0.504s passed
ALL_'field'_-1[hadoop2-tez] 0.512s passed
ALL_'field'_ALL[hadoop2-tez] 0.502s passed
ALL_'field'_REPLACE[hadoop2-tez] 0.054s passed
ALL_'field'_RESULTS[hadoop2-tez] 0.497s passed
ALL_'field'_SWAP[hadoop2-tez] 0.502s passed
ALL_ARGS_'field'[hadoop2-tez] 0.062s passed
ALL_ARGS_-1[hadoop2-tez] 0.489s passed
ALL_ARGS_ALL[hadoop2-tez] 0.059s passed
ALL_ARGS_REPLACE[hadoop2-tez] 0.457s passed
ALL_ARGS_RESULTS[hadoop2-tez] 0.429s passed
ALL_ARGS_SWAP[hadoop2-tez] 0.443s passed
ALL_UNKNOWN_'field'[hadoop2-tez] 0.043s passed
ALL_UNKNOWN_-1[hadoop2-tez] 0.557s passed
ALL_UNKNOWN_ALL[hadoop2-tez] 0.503s passed
ALL_UNKNOWN_REPLACE[hadoop2-tez] 0.491s passed
ALL_UNKNOWN_RESULTS[hadoop2-tez] 0.490s passed
ALL_UNKNOWN_SWAP[hadoop2-tez] 0.676s passed
ALL_VALUES_'field'[hadoop2-tez] 0.052s passed
ALL_VALUES_-1[hadoop2-tez] 0.439s passed
ALL_VALUES_ALL[hadoop2-tez] 0.428s passed
ALL_VALUES_REPLACE[hadoop2-tez] 0.445s passed
ALL_VALUES_RESULTS[hadoop2-tez] 0.437s passed
ALL_VALUES_SWAP[hadoop2-tez] 0.465s passed

Standard output

2016-01-14 17:16:15,189 INFO  platform.PlatformRunner (PlatformRunner.java:getPlatformClassesFromClasspath(195)) - found 1 test platforms from classpath
2016-01-14 17:16:15,219 INFO  platform.PlatformRunner (PlatformRunner.java:addPlatform(244)) - adding test: cascading.detail.EveryPipeAssemblyPlatformTest, with platform: hadoop2-tez
2016-01-14 17:16:15,693 INFO  tez.Hadoop2TezPlatform (Hadoop2TezPlatform.java:setUp(136)) - not using cluster
2016-01-14 17:16:15,987 WARN  util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2016-01-14 17:16:16,267 INFO  platform.TestPlatform (TestPlatform.java:getGlobalProperties(73)) - platform property overrides: 
2016-01-14 17:16:16,424 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:16,425 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:16,506 INFO  property.AppProps (AppProps.java:getAppID(177)) - using app.id: 51243B66928943F5916B1357871D5ECB
2016-01-14 17:16:16,716 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.085
2016-01-14 17:16:16,728 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.097
2016-01-14 17:16:16,729 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:16,729 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:16,731 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:16,957 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:17,106 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_-1
2016-01-14 17:16:17,304 INFO  util.Version (Version.java:printBanner(82)) - Concurrent, Inc - Cascading 3.0.4
2016-01-14 17:16:17,307 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:17,307 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:17,307 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_-1"]
2016-01-14 17:16:17,308 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:17,308 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:17,308 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:17,309 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...blyplatform/-1_'field'_-1
2016-01-14 17:16:17,309 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:17,332 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:17,507 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:17,513 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:17,513 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:17,560 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/4D810FCAE6244F93B1525990EED988AD/B33DBE7CC20D43FB950AD1815D48EC1B/.tez/application_1452791777512_0001 doesn't exist and is created
2016-01-14 17:16:17,640 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:17,640 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:17,641 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/4D810FCAE6244F93B1525990EED988AD/B33DBE7CC20D43FB950AD1815D48EC1B/.tez/application_1452791777512_0001
2016-01-14 17:16:17,682 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791777512_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:17,747 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: NEW
2016-01-14 17:16:17,747 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(238)) - DAGAppMaster is not started wait for 100ms...
2016-01-14 17:16:17,748 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:17,750 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:17,766 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:17,788 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:17,791 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:17,795 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/4D810FCAE6244F93B1525990EED988AD/B33DBE7CC20D43FB950AD1815D48EC1B/.tez/application_1452791777512_0001/localmode-log-dir
2016-01-14 17:16:17,796 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/4D810FCAE6244F93B1525990EED988AD/B33DBE7CC20D43FB950AD1815D48EC1B/.tez/application_1452791777512_0001/localmode-log-dir/history.txt.appattempt_1452791777512_0001_000000, maxErrors=10
2016-01-14 17:16:17,797 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:17,798 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791777512_0001_000000, appSubmitTime=1452791777661, launchTime=1452791777681
2016-01-14 17:16:17,805 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:17,806 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:17,813 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:17,813 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:17,848 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: INITED
2016-01-14 17:16:17,848 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(240)) - DAGAppMaster is not startetd wait for 100ms...
2016-01-14 17:16:17,848 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:17,865 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:55701
2016-01-14 17:16:17,906 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:17,949 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: INITED
2016-01-14 17:16:17,949 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(240)) - DAGAppMaster is not startetd wait for 100ms...
2016-01-14 17:16:17,949 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:17,981 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:17,982 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791777512_0001_000000, startTime=1452791777980
2016-01-14 17:16:17,982 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:18,049 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:18,066 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:18,066 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...blyplatform/-1_'field'_-1, applicationId=application_1452791777512_0001, dagName=[4D810FCAE6244F93B1525990EED988AD/B33DBE7CC20D43FB950AD1815D48EC1B] every/(1/1) ...blyplatform/-1_'field'_-1
2016-01-14 17:16:18,306 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [4D810FCAE6244F93B1525990EED988AD/B33DBE7CC20D43FB950AD1815D48EC1B] every/(1/1) ...blyplatform/-1_'field'_-1
2016-01-14 17:16:18,436 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791777512_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/4D810FCAE6244F93B1525990EED988AD/B33DBE7CC20D43FB950AD1815D48EC1B/.tez/application_1452791777512_0001/localmode-log-dir/dag_1452791777512_0001_1.dot
2016-01-14 17:16:18,440 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/4D810FCAE6244F93B1525990EED988AD/B33DBE7CC20D43FB950AD1815D48EC1B/.tez/application_1452791777512_0001/localmode-log-dir/dag_1452791777512_0001_1-tez-dag.pb.txt
2016-01-14 17:16:18,544 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791777512_0001_1
2016-01-14 17:16:18,544 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:18,545 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [4D810FCAE6244F93B1525990EED988AD/B33DBE7CC20D43FB950AD1815D48EC1B] every/(1/1) ...blyplatform/-1_'field'_-1
2016-01-14 17:16:18 Running Dag: dag_1452791777512_0001_1
2016-01-14 17:16:18,575 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:18,575 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791777512_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791777512_0001_1, submitTime=1452791778306
2016-01-14 17:16:18,612 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex F9A8D3DDEB9344A8A1E4B51DBC984F13
2016-01-14 17:16:18,620 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:18,623 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:18,623 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791777512_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791777512_0001_1, initTime=1452791778578
2016-01-14 17:16:18,624 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791777512_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:18,626 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:18,626 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791777512_0001_1][Event:DAG_STARTED]: dagID=dag_1452791777512_0001_1, startTime=1452791778626
2016-01-14 17:16:18,627 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:18,630 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791777512_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:18,630 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D5DD865D91094A5CB99C47A37A40F8AE : {439620531191490383E1B1852A21437B={InputName=439620531191490383E1B1852A21437B}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:18,631 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 439620531191490383E1B1852A21437B, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:18,631 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791777512_0001_1_00 [D5DD865D91094A5CB99C47A37A40F8AE]
2016-01-14 17:16:18,636 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...blyplatform/-1_'field'_-1, applicationId=application_1452791777512_0001, dagName=[4D810FCAE6244F93B1525990EED988AD/B33DBE7CC20D43FB950AD1815D48EC1B] every/(1/1) ...blyplatform/-1_'field'_-1
2016-01-14 17:16:18,638 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791777512_0001_1_00 [D5DD865D91094A5CB99C47A37A40F8AE]
2016-01-14 17:16:18,638 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791777512_0001_1_00 [D5DD865D91094A5CB99C47A37A40F8AE]
2016-01-14 17:16:18,640 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791777512_0001_1_00 [D5DD865D91094A5CB99C47A37A40F8AE]. Starting root input initializers: 1
2016-01-14 17:16:18,647 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 439620531191490383E1B1852A21437B on vertex vertex_1452791777512_0001_1_00 [D5DD865D91094A5CB99C47A37A40F8AE]
2016-01-14 17:16:18,659 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791777512_0001_1_00 [D5DD865D91094A5CB99C47A37A40F8AE] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:18,661 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791777512_0001_1_01 [F9A8D3DDEB9344A8A1E4B51DBC984F13]
2016-01-14 17:16:18,671 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791777512_0001, with dag id: dag_1452791777512_0001_1
2016-01-14 17:16:18,684 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:18,685 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791777512_0001_1_01 [F9A8D3DDEB9344A8A1E4B51DBC984F13]
2016-01-14 17:16:18,685 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:18,688 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 439620531191490383E1B1852A21437B asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:18,693 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:18,694 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:18,706 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791777512_0001_1_01 [F9A8D3DDEB9344A8A1E4B51DBC984F13]
2016-01-14 17:16:18,709 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791777512_0001_1_01 [F9A8D3DDEB9344A8A1E4B51DBC984F13]
2016-01-14 17:16:18,709 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=90EB992F64BB43B8890CBF0E79AFB3E0, vertexId=vertex_1452791777512_0001_1_01 [F9A8D3DDEB9344A8A1E4B51DBC984F13], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:18,741 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=90EB992F64BB43B8890CBF0E79AFB3E0, vertexId=vertex_1452791777512_0001_1_01 [F9A8D3DDEB9344A8A1E4B51DBC984F13]
2016-01-14 17:16:18,763 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=90EB992F64BB43B8890CBF0E79AFB3E0, vertexName=F9A8D3DDEB9344A8A1E4B51DBC984F13, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:18,776 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=90EB992F64BB43B8890CBF0E79AFB3E0, vertexId=vertex_1452791777512_0001_1_01 [F9A8D3DDEB9344A8A1E4B51DBC984F13]
2016-01-14 17:16:18,777 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:18,780 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 439620531191490383E1B1852A21437B on vertex vertex_1452791777512_0001_1_00 [D5DD865D91094A5CB99C47A37A40F8AE]
2016-01-14 17:16:18,781 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:18,782 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791777512_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F9A8D3DDEB9344A8A1E4B51DBC984F13, vertexId=vertex_1452791777512_0001_1_01, initRequestedTime=1452791778659, initedTime=1452791778779, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:18,782 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791777512_0001_1_01 [F9A8D3DDEB9344A8A1E4B51DBC984F13] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:18,784 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791777512_0001_1_00 [D5DD865D91094A5CB99C47A37A40F8AE] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:18,784 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {439620531191490383E1B1852A21437B=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:18,785 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791777512_0001_1_00 [D5DD865D91094A5CB99C47A37A40F8AE] parallelism set to 1
2016-01-14 17:16:18,785 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791777512_0001_1_00 [D5DD865D91094A5CB99C47A37A40F8AE] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:18,799 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791777512_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791777512_0001_1_00, eventCount=1
2016-01-14 17:16:18,800 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:18,800 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791777512_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D5DD865D91094A5CB99C47A37A40F8AE, vertexId=vertex_1452791777512_0001_1_00, initRequestedTime=1452791778630, initedTime=1452791778799, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:18,800 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791777512_0001_1_00 [D5DD865D91094A5CB99C47A37A40F8AE] with distanceFromRoot: 0
2016-01-14 17:16:18,800 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791777512_0001_1_00 [D5DD865D91094A5CB99C47A37A40F8AE] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:18,801 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D5DD865D91094A5CB99C47A37A40F8AE
2016-01-14 17:16:18,801 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791777512_0001_1_00 [D5DD865D91094A5CB99C47A37A40F8AE]
2016-01-14 17:16:18,803 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:18,803 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791777512_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791777512_0001_1_00, startRequestedTime=1452791778801, startedTime=1452791778801
2016-01-14 17:16:18,805 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791777512_0001_1_00 [D5DD865D91094A5CB99C47A37A40F8AE] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:18,831 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:18,832 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791777512_0001_1][Event:TASK_STARTED]: vertexName=D5DD865D91094A5CB99C47A37A40F8AE, taskId=task_1452791777512_0001_1_00_000000, scheduledTime=1452791778830, launchTime=1452791778830
2016-01-14 17:16:18,832 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791777512_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:18,837 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791777512_0001_1_00 for vertex: vertex_1452791777512_0001_1_01 [F9A8D3DDEB9344A8A1E4B51DBC984F13] numStartedSources: 1 numSources: 1
2016-01-14 17:16:18,837 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791777512_0001_1_01 [F9A8D3DDEB9344A8A1E4B51DBC984F13] with distanceFromRoot: 1
2016-01-14 17:16:18,839 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791777512_0001_1_00 [D5DD865D91094A5CB99C47A37A40F8AE]
2016-01-14 17:16:18,843 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: D5DD865D91094A5CB99C47A37A40F8AE in vertex: F9A8D3DDEB9344A8A1E4B51DBC984F13
2016-01-14 17:16:18,846 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: F9A8D3DDEB9344A8A1E4B51DBC984F13 with 1 source tasks and 1 pending tasks
2016-01-14 17:16:18,847 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:18,847 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791777512_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791777512_0001_1_01, startRequestedTime=1452791778839, startedTime=1452791778839
2016-01-14 17:16:18,847 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791777512_0001_1_01 [F9A8D3DDEB9344A8A1E4B51DBC984F13] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:18,850 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [4D810FCAE6244F93B1525990EED988AD/B33DBE7CC20D43FB950AD1815D48EC1B] every/(1/1) ...blyplatform/-1_'field'_-1, VertexName: D5DD865D91094A5CB99C47A37A40F8AE, VertexParallelism: 1, TaskAttemptID:attempt_1452791777512_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=439620531191490383E1B1852A21437B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F9A8D3DDEB9344A8A1E4B51DBC984F13, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:18,854 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791777512_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:18,855 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:18,903 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:18,907 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:18,908 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:18,912 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791777512_0001_00_000001
2016-01-14 17:16:18,912 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791777512_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:18,921 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791777512_0001_00_000001
2016-01-14 17:16:18,921 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:18,926 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791777512_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:18,927 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:18,928 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791777512_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791777512_0001_00_000001, launchTime=1452791778926
2016-01-14 17:16:18,931 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791777512_0001_00_000001
2016-01-14 17:16:18,938 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791777512_0001_1_00_000000_0] to container: [container_1452791777512_0001_00_000001]
2016-01-14 17:16:18,939 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791777512_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:18,940 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791777512_0001_00_000001 given task: attempt_1452791777512_0001_1_00_000000_0
2016-01-14 17:16:18,941 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791777512_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791777512_0001_1_00_000000_0
2016-01-14 17:16:18,942 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791777512_0001_1_00_000000_0] started. Is using containerId: [container_1452791777512_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:18 Starting to run new task attempt: attempt_1452791777512_0001_1_00_000000_0
2016-01-14 17:16:18,943 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791777512_0001_1_00_000000_0
2016-01-14 17:16:18,945 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:18,943 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791777512_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D5DD865D91094A5CB99C47A37A40F8AE, taskAttemptId=attempt_1452791777512_0001_1_00_000000_0, startTime=1452791778940, containerId=container_1452791777512_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791777512_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:18,946 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791777512_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:18,946 INFO  counters.Limits (Limits.java:ensureInitialized(59)) - Counter limits initialized with parameters:  GROUP_NAME_MAX=128, MAX_GROUPS=500, COUNTER_NAME_MAX=64, MAX_COUNTERS=1200
2016-01-14 17:16:18,950 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:18,952 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791777512_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:18,952 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:18,953 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:18,953 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:18,955 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:18,983 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:18,984 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [4D810FCAE6244F93B1525990EED988AD/B33DBE7CC20D43FB950AD1815D48EC1B] every/(1/1) ...blyplatform/-1_'field'_-1, VertexName: D5DD865D91094A5CB99C47A37A40F8AE, VertexParallelism: 1, TaskAttemptID:attempt_1452791777512_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=439620531191490383E1B1852A21437B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F9A8D3DDEB9344A8A1E4B51DBC984F13, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:18,986 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:16:18,990 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791777512_0001_1_00_000000_0
2016-01-14 17:16:18,990 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:18,991 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:18,992 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791777512_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:18,996 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=439620531191490383E1B1852A21437B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:18,997 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:18,997 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F9A8D3DDEB9344A8A1E4B51DBC984F13, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:18,998 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:18,998 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 439620531191490383E1B1852A21437B
2016-01-14 17:16:19,001 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:19,002 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 439620531191490383E1B1852A21437B
2016-01-14 17:16:19,009 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F9A8D3DDEB9344A8A1E4B51DBC984F13
2016-01-14 17:16:19,015 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:19,020 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:19,020 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 439620531191490383E1B1852A21437B
2016-01-14 17:16:19,021 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 439620531191490383E1B1852A21437B
2016-01-14 17:16:19,026 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:19,027 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: F9A8D3DDEB9344A8A1E4B51DBC984F13
2016-01-14 17:16:19,027 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F9A8D3DDEB9344A8A1E4B51DBC984F13
2016-01-14 17:16:19,034 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D5DD865D91094A5CB99C47A37A40F8AE, ordinal: 0
2016-01-14 17:16:19,035 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D5DD865D91094A5CB99C47A37A40F8AE, mem on start (mb), free: 242, total: 281, max: 672
2016-01-14 17:16:19,036 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:19,036 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:19,036 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:19,037 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:19,037 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:19,042 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:19,042 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:19,042 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:19,043 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:19,043 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 17:16:19,044 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 439620531191490383E1B1852A21437B, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:19,044 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F9A8D3DDEB9344A8A1E4B51DBC984F13, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:19,044 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:19,045 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 439620531191490383E1B1852A21437B
2016-01-14 17:16:19,045 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:19,045 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 439620531191490383E1B1852A21437B
2016-01-14 17:16:19,046 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 439620531191490383E1B1852A21437B being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:19,047 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:19,047 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:19,048 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:19,050 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791777512_0001_1_00_000000_0
2016-01-14 17:16:19,053 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:19,059 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@3845ba27
2016-01-14 17:16:19,071 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@3845ba27
2016-01-14 17:16:19,072 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:19,072 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:19,087 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:19,101 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 439620531191490383E1B1852A21437B
2016-01-14 17:16:19,101 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 028392ABB8CD47BEBB6C86FCCC93D79A
2016-01-14 17:16:19,102 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 028392ABB8CD47BEBB6C86FCCC93D79A
2016-01-14 17:16:19,122 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@2c160110; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@589fd80c; comparator=cascading.tuple.hadoop.util.TupleComparator@b1d6a5a; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:19,126 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:19,128 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:19,129 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:19,257 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:19,258 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:19,259 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:19,259 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:19,259 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:19,266 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:19,266 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:19,268 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D5DD865D91094A5CB99C47A37A40F8AE, all 1 inputs ready in: 00:00:00.002
2016-01-14 17:16:19,288 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791777512_0001_1_00_000000_0
2016-01-14 17:16:19,288 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D5DD865D91094A5CB99C47A37A40F8AE, mem on close (mb), free: 132, total: 281, max: 672
2016-01-14 17:16:19,289 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:19,289 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:19,289 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:19,289 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:19,305 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:19 Completed running task attempt: attempt_1452791777512_0001_1_00_000000_0
2016-01-14 17:16:19,330 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791777512_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:19,331 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791777512_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791777512_0001_1_00, eventCount=1
2016-01-14 17:16:19,335 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791779268, Process_Duration=20, Process_End_Time=1452791779288, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=3][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:19,336 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:19,337 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:19,338 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:19,338 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791777512_0001_00_000001
2016-01-14 17:16:19,339 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791777512_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D5DD865D91094A5CB99C47A37A40F8AE, taskAttemptId=attempt_1452791777512_0001_1_00_000000_0, startTime=1452791778940, finishTime=1452791779332, timeTaken=392, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791779268, Process_Duration=20, Process_End_Time=1452791779288, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=3, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:19,339 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791777512_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:19,340 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:19,343 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791777512_0001_1_00_000000_0
2016-01-14 17:16:19,346 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:19,349 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791777512_0001_1][Event:TASK_FINISHED]: vertexName=D5DD865D91094A5CB99C47A37A40F8AE, taskId=task_1452791777512_0001_1_00_000000, startTime=1452791778940, finishTime=1452791779345, timeTaken=405, status=SUCCEEDED, successfulAttemptID=attempt_1452791777512_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791779268, Process_Duration=20, Process_End_Time=1452791779288, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=3, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:19,350 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791777512_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:19,353 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791777512_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:19,354 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791777512_0001_1_00 [D5DD865D91094A5CB99C47A37A40F8AE] : 1
2016-01-14 17:16:19,355 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791777512_0001_1_00 [D5DD865D91094A5CB99C47A37A40F8AE]
2016-01-14 17:16:19,363 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:19,366 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791777512_0001_1][Event:VERTEX_FINISHED]: vertexName=D5DD865D91094A5CB99C47A37A40F8AE, vertexId=vertex_1452791777512_0001_1_00, initRequestedTime=1452791778630, initedTime=1452791778799, startRequestedTime=1452791778801, startedTime=1452791778801, finishTime=1452791779355, timeTaken=554, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791779268, Process_Duration=20, Process_End_Time=1452791779288, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=3, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791778940, firstTasksToStart=[ task_1452791777512_0001_1_00_000000 ], lastTaskFinishTime=1452791779332, lastTasksToFinish=[ task_1452791777512_0001_1_00_000000 ], minTaskDuration=392, maxTaskDuration=392, avgTaskDuration=392.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791777512_0001_1_00_000000 ], longestDurationTasks=[ task_1452791777512_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:19,366 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791777512_0001_1_00 [D5DD865D91094A5CB99C47A37A40F8AE] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:19,370 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:19,371 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791777512_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791777512_0001_00_000001, stoppedTime=1452791779370, exitStatus=0
2016-01-14 17:16:19,372 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791777512_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:19,372 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791777512_0001_1_01 [F9A8D3DDEB9344A8A1E4B51DBC984F13] attempt: attempt_1452791777512_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:19,373 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: F9A8D3DDEB9344A8A1E4B51DBC984F13
2016-01-14 17:16:19,373 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791777512_0001_1_01 [F9A8D3DDEB9344A8A1E4B51DBC984F13]
2016-01-14 17:16:19,375 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791777512_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:19,376 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791777512_0001_00_000001: 31 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:19,376 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791777512_0001_00_000001, Exiting
2016-01-14 17:16:19,376 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791777512_0001_00_000001
2016-01-14 17:16:19,376 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791777512_0001_00_000001
2016-01-14 17:16:19,376 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791777512_0001_1_00 [D5DD865D91094A5CB99C47A37A40F8AE] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:19,377 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791777512_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:19,377 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791777512_0001_1][Event:TASK_STARTED]: vertexName=F9A8D3DDEB9344A8A1E4B51DBC984F13, taskId=task_1452791777512_0001_1_01_000000, scheduledTime=1452791779377, launchTime=1452791779377
2016-01-14 17:16:19,378 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791777512_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:19,378 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:19,382 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791777512_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:19,382 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791777512_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:19,384 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791777512_0001_00_000001 completed successfully
2016-01-14 17:16:19,383 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791777512_0001_1_01 [F9A8D3DDEB9344A8A1E4B51DBC984F13]
2016-01-14 17:16:19,386 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [4D810FCAE6244F93B1525990EED988AD/B33DBE7CC20D43FB950AD1815D48EC1B] every/(1/1) ...blyplatform/-1_'field'_-1, VertexName: F9A8D3DDEB9344A8A1E4B51DBC984F13, VertexParallelism: 1, TaskAttemptID:attempt_1452791777512_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D5DD865D91094A5CB99C47A37A40F8AE, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=90EB992F64BB43B8890CBF0E79AFB3E0, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:19,387 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791777512_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:19,387 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791777512_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:19,387 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:19,389 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791777512_0001_00_000002
2016-01-14 17:16:19,389 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791777512_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:19,393 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791777512_0001_00_000002
2016-01-14 17:16:19,394 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 17:16:19,398 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:19,398 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791777512_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791777512_0001_00_000002, launchTime=1452791779398
2016-01-14 17:16:19,399 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791777512_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:19,400 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791777512_0001_00_000002
2016-01-14 17:16:19,403 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791777512_0001_1_01_000000_0] to container: [container_1452791777512_0001_00_000002]
2016-01-14 17:16:19,403 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791777512_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:19,404 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791777512_0001_1_01_000000_0] started. Is using containerId: [container_1452791777512_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:19,404 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:19,405 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791777512_0001_00_000002 given task: attempt_1452791777512_0001_1_01_000000_0
2016-01-14 17:16:19,405 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791777512_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791777512_0001_1_01_000000_0
2016-01-14 17:16:19,405 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791777512_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F9A8D3DDEB9344A8A1E4B51DBC984F13, taskAttemptId=attempt_1452791777512_0001_1_01_000000_0, startTime=1452791779403, containerId=container_1452791777512_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791777512_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:19,405 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791777512_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:19 Starting to run new task attempt: attempt_1452791777512_0001_1_01_000000_0
2016-01-14 17:16:19,406 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791777512_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:19,406 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791777512_0001_1_01_000000_0
2016-01-14 17:16:19,408 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:19,408 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:19,408 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:19,409 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:19,409 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:19,409 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:19,411 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [4D810FCAE6244F93B1525990EED988AD/B33DBE7CC20D43FB950AD1815D48EC1B] every/(1/1) ...blyplatform/-1_'field'_-1, VertexName: F9A8D3DDEB9344A8A1E4B51DBC984F13, VertexParallelism: 1, TaskAttemptID:attempt_1452791777512_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D5DD865D91094A5CB99C47A37A40F8AE, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=90EB992F64BB43B8890CBF0E79AFB3E0, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:19,411 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:16:19,413 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791777512_0001_1_01_000000_0
2016-01-14 17:16:19,417 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:19,417 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:19,416 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791777512_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:19,418 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:19,418 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=90EB992F64BB43B8890CBF0E79AFB3E0, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:19,419 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:19,420 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 90EB992F64BB43B8890CBF0E79AFB3E0
2016-01-14 17:16:19,420 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:19,421 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 90EB992F64BB43B8890CBF0E79AFB3E0
2016-01-14 17:16:19,424 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:19,426 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=D5DD865D91094A5CB99C47A37A40F8AE, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:19,428 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:19,429 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: D5DD865D91094A5CB99C47A37A40F8AE
2016-01-14 17:16:19,444 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F9A8D3DDEB9344A8A1E4B51DBC984F13, ordinal: 1
2016-01-14 17:16:19,447 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F9A8D3DDEB9344A8A1E4B51DBC984F13, mem on start (mb), free: 121, total: 281, max: 672
2016-01-14 17:16:19,447 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:19,447 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:19,451 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:19,452 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 90EB992F64BB43B8890CBF0E79AFB3E0
2016-01-14 17:16:19,452 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:19,453 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:19,453 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:19,453 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: D5DD865D91094A5CB99C47A37A40F8AE
2016-01-14 17:16:19,454 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: D5DD865D91094A5CB99C47A37A40F8AE
2016-01-14 17:16:19,454 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:19,454 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:19,454 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:19,455 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:19,455 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:19,455 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:19,455 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:19,455 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 90EB992F64BB43B8890CBF0E79AFB3E0, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:19,455 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, D5DD865D91094A5CB99C47A37A40F8AE, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 17:16:19,456 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:19,456 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: D5DD865D91094A5CB99C47A37A40F8AE
2016-01-14 17:16:19,456 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: D5DD865D91094A5CB99C47A37A40F8AE being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:19,456 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:19,456 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:19,466 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:19,466 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:19,471 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: D5DD865D91094A5CB99C47A37A40F8AE with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:19,473 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 17:16:19,474 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:19,478 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:19,481 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: D5DD865D91094A5CB99C47A37A40F8AE
2016-01-14 17:16:19,481 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:19,481 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791777512_0001_1_01_000000_0
2016-01-14 17:16:19,482 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791777512_0001_1_00_000000_0_10003, runDuration: 302157, ]
2016-01-14 17:16:19,487 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:19,494 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791777512_0001_1_00_000000_0_10003] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791779494, TimeTaken=2, Rate=0.14 MB/s
2016-01-14 17:16:19,494 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : D5DD865D91094A5CB99C47A37A40F8AE
2016-01-14 17:16:19,495 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:19,496 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [D5DD865D91094A5CB99C47A37A40F8AE] #1 in 4ms
2016-01-14 17:16:19,497 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [D5DD865D91094A5CB99C47A37A40F8AE] #1, status:false, isInterrupted:false
2016-01-14 17:16:19,498 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:19,501 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:19,502 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:19,507 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:19,508 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 028392ABB8CD47BEBB6C86FCCC93D79A
2016-01-14 17:16:19,511 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_-1"], id: 90EB992F64BB43B8890CBF0E79AFB3E0
2016-01-14 17:16:19,511 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:19,512 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : D5DD865D91094A5CB99C47A37A40F8AE
2016-01-14 17:16:19,511 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_-1"]
2016-01-14 17:16:19,512 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:19,513 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 028392ABB8CD47BEBB6C86FCCC93D79A, for 1 inputs
2016-01-14 17:16:19,515 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F9A8D3DDEB9344A8A1E4B51DBC984F13, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:19,518 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@3eefc773; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@700f6f4c
2016-01-14 17:16:19,528 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:19,529 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:19,530 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791777512_0001_1_01_000000_0
2016-01-14 17:16:19,531 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791777512_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:19,532 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791777512_0001_1_01_000000_0
2016-01-14 17:16:19,532 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F9A8D3DDEB9344A8A1E4B51DBC984F13, mem on close (mb), free: 113, total: 281, max: 672
2016-01-14 17:16:19,532 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: D5DD865D91094A5CB99C47A37A40F8AE
2016-01-14 17:16:19,533 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:19,533 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:19,534 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:19,534 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:19,534 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:19 Completed running task attempt: attempt_1452791777512_0001_1_01_000000_0
2016-01-14 17:16:19,536 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791777512_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:19,538 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:19,540 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=30, MERGE_PHASE_TIME=46, FIRST_EVENT_RECEIVED=26, LAST_EVENT_RECEIVED=26][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791779516, Process_Duration=16, Process_End_Time=1452791779532, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 17:16:19,541 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:19,541 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:19,542 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791777512_0001_00_000002
2016-01-14 17:16:19,545 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791777512_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F9A8D3DDEB9344A8A1E4B51DBC984F13, taskAttemptId=attempt_1452791777512_0001_1_01_000000_0, startTime=1452791779403, finishTime=1452791779538, timeTaken=135, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=30, MERGE_PHASE_TIME=46, FIRST_EVENT_RECEIVED=26, LAST_EVENT_RECEIVED=26, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791779516, Process_Duration=16, Process_End_Time=1452791779532, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:19,545 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791777512_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:19,545 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791777512_0001_1_01_000000_0
2016-01-14 17:16:19,545 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:19,546 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:19,547 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791777512_0001_1][Event:TASK_FINISHED]: vertexName=F9A8D3DDEB9344A8A1E4B51DBC984F13, taskId=task_1452791777512_0001_1_01_000000, startTime=1452791779403, finishTime=1452791779545, timeTaken=142, status=SUCCEEDED, successfulAttemptID=attempt_1452791777512_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=30, MERGE_PHASE_TIME=46, FIRST_EVENT_RECEIVED=26, LAST_EVENT_RECEIVED=26, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791779516, Process_Duration=16, Process_End_Time=1452791779532, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:19,547 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791777512_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:19,548 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791777512_0001_1_01 [F9A8D3DDEB9344A8A1E4B51DBC984F13] : 1
2016-01-14 17:16:19,548 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791777512_0001_1_01 [F9A8D3DDEB9344A8A1E4B51DBC984F13]
2016-01-14 17:16:19,549 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:19,550 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791777512_0001_1][Event:VERTEX_FINISHED]: vertexName=F9A8D3DDEB9344A8A1E4B51DBC984F13, vertexId=vertex_1452791777512_0001_1_01, initRequestedTime=1452791778659, initedTime=1452791778779, startRequestedTime=1452791778839, startedTime=1452791778839, finishTime=1452791779548, timeTaken=709, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=30, MERGE_PHASE_TIME=46, FIRST_EVENT_RECEIVED=26, LAST_EVENT_RECEIVED=26, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791779516, Process_Duration=16, Process_End_Time=1452791779532, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452791779403, firstTasksToStart=[ task_1452791777512_0001_1_01_000000 ], lastTaskFinishTime=1452791779538, lastTasksToFinish=[ task_1452791777512_0001_1_01_000000 ], minTaskDuration=135, maxTaskDuration=135, avgTaskDuration=135.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791777512_0001_1_01_000000 ], longestDurationTasks=[ task_1452791777512_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:19,551 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791777512_0001_1_01 [F9A8D3DDEB9344A8A1E4B51DBC984F13] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:19,551 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791777512_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:19,551 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791777512_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791777512_0001_00_000002, stoppedTime=1452791779551, exitStatus=0
2016-01-14 17:16:19,551 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:19,552 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791777512_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:19,552 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791777512_0001_1_01 [F9A8D3DDEB9344A8A1E4B51DBC984F13] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:19,553 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791777512_0001_1
2016-01-14 17:16:19,553 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791777512_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 17:16:19,554 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791777512_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:19,554 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791777512_0001_00_000002, Exiting
2016-01-14 17:16:19,554 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791777512_0001_00_000002
2016-01-14 17:16:19,554 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791777512_0001_00_000002
2016-01-14 17:16:19,555 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791777512_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452791777512_0001_1
2016-01-14 17:16:19,555 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452791777512_0001_1_00 [D5DD865D91094A5CB99C47A37A40F8AE]
2016-01-14 17:16:19,555 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 90EB992F64BB43B8890CBF0E79AFB3E0 for vertex: vertex_1452791777512_0001_1_01 [F9A8D3DDEB9344A8A1E4B51DBC984F13]
2016-01-14 17:16:19,560 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791777512_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:19,560 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791777512_0001_00_000002 completed successfully
2016-01-14 17:16:19,577 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791777512_0001_1, queueSize=0
2016-01-14 17:16:19,577 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:19,578 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791777512_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791777512_0001_1, startTime=1452791778626, finishTime=1452791779570, timeTaken=944, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=465, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=590348288, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=30, MERGE_PHASE_TIME=46, FIRST_EVENT_RECEIVED=26, LAST_EVENT_RECEIVED=26, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905583558784, Process_Duration=36, Process_End_Time=2905583558820, Read_Duration=5, Tuples_Read=40, Tuples_Written=40, Write_Duration=4, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 17:16:19,579 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791777512_0001_1 finished with state: SUCCEEDED
2016-01-14 17:16:19,579 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791777512_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:19,580 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452791777512_0001_00_000002
2016-01-14 17:16:19,580 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791777512_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:16:19 Completed Dag: dag_1452791777512_0001_1
2016-01-14 17:16:19,582 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791777512_0001_1, dagState=SUCCEEDED
2016-01-14 17:16:19,582 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791777512_0001_1_post
2016-01-14 17:16:19,582 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:19,583 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:19,591 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...blyplatform/-1_'field'_-1, applicationId=application_1452791777512_0001
2016-01-14 17:16:19,594 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:19,594 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:19,595 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:19,595 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:19,595 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:19,596 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:19,598 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:19,598 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:19,598 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:19,599 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:19,599 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:19,600 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:19,600 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:19,606 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/4D810FCAE6244F93B1525990EED988AD/B33DBE7CC20D43FB950AD1815D48EC1B/.tez/application_1452791777512_0001
2016-01-14 17:16:19,607 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:02.300
2016-01-14 17:16:19,705 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_'field'_-1
2016-01-14 17:16:19,746 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:19,747 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:19,836 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.041
2016-01-14 17:16:19,858 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.063
2016-01-14 17:16:19,859 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:19,860 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:19,860 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:19,887 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:19,906 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_'field'
2016-01-14 17:16:20,007 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:20,008 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:20,008 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_'field'"]
2016-01-14 17:16:20,009 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:20,009 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:20,010 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:20,010 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...atform/-1_'field'_'field'
2016-01-14 17:16:20,011 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:20,035 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:20,041 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:20,043 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:20,043 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:20,055 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/15F66FE867924B1D9BC00090E5FD561A/B83A7021DDCA4D2BBF4F120F1AD9C007/.tez/application_1452791780042_0001 doesn't exist and is created
2016-01-14 17:16:20,122 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:20,122 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:20,123 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/15F66FE867924B1D9BC00090E5FD561A/B83A7021DDCA4D2BBF4F120F1AD9C007/.tez/application_1452791780042_0001
2016-01-14 17:16:20,129 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791780042_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:20,145 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:20,146 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:20,149 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:20,149 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:20,150 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/15F66FE867924B1D9BC00090E5FD561A/B83A7021DDCA4D2BBF4F120F1AD9C007/.tez/application_1452791780042_0001/localmode-log-dir
2016-01-14 17:16:20,150 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/15F66FE867924B1D9BC00090E5FD561A/B83A7021DDCA4D2BBF4F120F1AD9C007/.tez/application_1452791780042_0001/localmode-log-dir/history.txt.appattempt_1452791780042_0001_000000, maxErrors=10
2016-01-14 17:16:20,151 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:20,152 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791780042_0001_000000, appSubmitTime=1452791780128, launchTime=1452791780128
2016-01-14 17:16:20,157 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:20,162 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:20,165 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:20,165 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:20,171 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:50459
2016-01-14 17:16:20,175 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:20,178 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791780042_0001_000000, startTime=1452791780178
2016-01-14 17:16:20,178 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:20,178 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:20,222 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:20,222 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:20,223 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...atform/-1_'field'_'field', applicationId=application_1452791780042_0001, dagName=[15F66FE867924B1D9BC00090E5FD561A/B83A7021DDCA4D2BBF4F120F1AD9C007] every/(1/1) ...atform/-1_'field'_'field'
2016-01-14 17:16:20,250 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [15F66FE867924B1D9BC00090E5FD561A/B83A7021DDCA4D2BBF4F120F1AD9C007] every/(1/1) ...atform/-1_'field'_'field'
2016-01-14 17:16:20,254 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791780042_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/15F66FE867924B1D9BC00090E5FD561A/B83A7021DDCA4D2BBF4F120F1AD9C007/.tez/application_1452791780042_0001/localmode-log-dir/dag_1452791780042_0001_1.dot
2016-01-14 17:16:20,255 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/15F66FE867924B1D9BC00090E5FD561A/B83A7021DDCA4D2BBF4F120F1AD9C007/.tez/application_1452791780042_0001/localmode-log-dir/dag_1452791780042_0001_1-tez-dag.pb.txt
2016-01-14 17:16:20,283 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791780042_0001_1
2016-01-14 17:16:20,283 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:20,283 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [15F66FE867924B1D9BC00090E5FD561A/B83A7021DDCA4D2BBF4F120F1AD9C007] every/(1/1) ...atform/-1_'field'_'field'
2016-01-14 17:16:20 Running Dag: dag_1452791780042_0001_1
2016-01-14 17:16:20,299 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791780042_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791780042_0001_1, submitTime=1452791780250
2016-01-14 17:16:20,299 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:20,300 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex B221A9D82BB34B3CA5299E28B4CD65BD
2016-01-14 17:16:20,301 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:20,301 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:20,301 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791780042_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791780042_0001_1, initTime=1452791780299
2016-01-14 17:16:20,304 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791780042_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:20,304 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:20,304 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791780042_0001_1][Event:DAG_STARTED]: dagID=dag_1452791780042_0001_1, startTime=1452791780304
2016-01-14 17:16:20,304 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:20,305 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791780042_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:20,305 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: F60C4FE5443841C7BAD85BA367A002A4 : {B9EAEDE1EDCF42DFA7668F1024E7D137={InputName=B9EAEDE1EDCF42DFA7668F1024E7D137}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:20,305 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: B9EAEDE1EDCF42DFA7668F1024E7D137, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:20,305 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791780042_0001_1_00 [F60C4FE5443841C7BAD85BA367A002A4]
2016-01-14 17:16:20,305 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791780042_0001_1_00 [F60C4FE5443841C7BAD85BA367A002A4]
2016-01-14 17:16:20,305 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791780042_0001_1_00 [F60C4FE5443841C7BAD85BA367A002A4]
2016-01-14 17:16:20,306 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791780042_0001_1_00 [F60C4FE5443841C7BAD85BA367A002A4]. Starting root input initializers: 1
2016-01-14 17:16:20,306 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791780042_0001_1_00 [F60C4FE5443841C7BAD85BA367A002A4] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:20,306 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: B9EAEDE1EDCF42DFA7668F1024E7D137 on vertex vertex_1452791780042_0001_1_00 [F60C4FE5443841C7BAD85BA367A002A4]
2016-01-14 17:16:20,307 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...atform/-1_'field'_'field', applicationId=application_1452791780042_0001, dagName=[15F66FE867924B1D9BC00090E5FD561A/B83A7021DDCA4D2BBF4F120F1AD9C007] every/(1/1) ...atform/-1_'field'_'field'
2016-01-14 17:16:20,306 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791780042_0001_1_01 [B221A9D82BB34B3CA5299E28B4CD65BD]
2016-01-14 17:16:20,311 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:20,311 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input B9EAEDE1EDCF42DFA7668F1024E7D137 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:20,312 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:20,312 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:20,315 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:20,316 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: B9EAEDE1EDCF42DFA7668F1024E7D137 on vertex vertex_1452791780042_0001_1_00 [F60C4FE5443841C7BAD85BA367A002A4]
2016-01-14 17:16:20,317 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:20,317 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791780042_0001_1_01 [B221A9D82BB34B3CA5299E28B4CD65BD]
2016-01-14 17:16:20,317 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791780042_0001_1_01 [B221A9D82BB34B3CA5299E28B4CD65BD]
2016-01-14 17:16:20,317 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791780042_0001_1_01 [B221A9D82BB34B3CA5299E28B4CD65BD]
2016-01-14 17:16:20,318 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=3F04FC8D96D947FDAD68E90C70A6D222, vertexId=vertex_1452791780042_0001_1_01 [B221A9D82BB34B3CA5299E28B4CD65BD], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:20,318 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=3F04FC8D96D947FDAD68E90C70A6D222, vertexId=vertex_1452791780042_0001_1_01 [B221A9D82BB34B3CA5299E28B4CD65BD]
2016-01-14 17:16:20,321 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=3F04FC8D96D947FDAD68E90C70A6D222, vertexName=B221A9D82BB34B3CA5299E28B4CD65BD, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:20,321 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=3F04FC8D96D947FDAD68E90C70A6D222, vertexId=vertex_1452791780042_0001_1_01 [B221A9D82BB34B3CA5299E28B4CD65BD]
2016-01-14 17:16:20,323 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:20,323 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791780042_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B221A9D82BB34B3CA5299E28B4CD65BD, vertexId=vertex_1452791780042_0001_1_01, initRequestedTime=1452791780306, initedTime=1452791780323, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:20,323 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791780042_0001_1_01 [B221A9D82BB34B3CA5299E28B4CD65BD] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:20,324 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791780042_0001_1_00 [F60C4FE5443841C7BAD85BA367A002A4] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:20,324 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {B9EAEDE1EDCF42DFA7668F1024E7D137=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:20,324 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791780042_0001_1_00 [F60C4FE5443841C7BAD85BA367A002A4] parallelism set to 1
2016-01-14 17:16:20,325 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791780042_0001_1_00 [F60C4FE5443841C7BAD85BA367A002A4] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:20,325 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791780042_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791780042_0001_1_00, eventCount=1
2016-01-14 17:16:20,327 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791780042_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F60C4FE5443841C7BAD85BA367A002A4, vertexId=vertex_1452791780042_0001_1_00, initRequestedTime=1452791780305, initedTime=1452791780325, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:20,327 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791780042_0001_1_00 [F60C4FE5443841C7BAD85BA367A002A4] with distanceFromRoot: 0
2016-01-14 17:16:20,327 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791780042_0001, with dag id: dag_1452791780042_0001_1
2016-01-14 17:16:20,327 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:20,328 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791780042_0001_1_00 [F60C4FE5443841C7BAD85BA367A002A4] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:20,329 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in F60C4FE5443841C7BAD85BA367A002A4
2016-01-14 17:16:20,329 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791780042_0001_1_00 [F60C4FE5443841C7BAD85BA367A002A4]
2016-01-14 17:16:20,330 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:20,330 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791780042_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791780042_0001_1_00, startRequestedTime=1452791780329, startedTime=1452791780329
2016-01-14 17:16:20,330 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791780042_0001_1_00 [F60C4FE5443841C7BAD85BA367A002A4] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:20,331 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:20,332 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791780042_0001_1][Event:TASK_STARTED]: vertexName=F60C4FE5443841C7BAD85BA367A002A4, taskId=task_1452791780042_0001_1_00_000000, scheduledTime=1452791780331, launchTime=1452791780331
2016-01-14 17:16:20,332 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791780042_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:20,334 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791780042_0001_1_00 for vertex: vertex_1452791780042_0001_1_01 [B221A9D82BB34B3CA5299E28B4CD65BD] numStartedSources: 1 numSources: 1
2016-01-14 17:16:20,337 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791780042_0001_1_01 [B221A9D82BB34B3CA5299E28B4CD65BD] with distanceFromRoot: 1
2016-01-14 17:16:20,339 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791780042_0001_1_00 [F60C4FE5443841C7BAD85BA367A002A4]
2016-01-14 17:16:20,340 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: F60C4FE5443841C7BAD85BA367A002A4 in vertex: B221A9D82BB34B3CA5299E28B4CD65BD
2016-01-14 17:16:20,341 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: B221A9D82BB34B3CA5299E28B4CD65BD with 1 source tasks and 1 pending tasks
2016-01-14 17:16:20,341 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:20,342 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791780042_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791780042_0001_1_01, startRequestedTime=1452791780339, startedTime=1452791780339
2016-01-14 17:16:20,342 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791780042_0001_1_01 [B221A9D82BB34B3CA5299E28B4CD65BD] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:20,343 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [15F66FE867924B1D9BC00090E5FD561A/B83A7021DDCA4D2BBF4F120F1AD9C007] every/(1/1) ...atform/-1_'field'_'field', VertexName: F60C4FE5443841C7BAD85BA367A002A4, VertexParallelism: 1, TaskAttemptID:attempt_1452791780042_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B9EAEDE1EDCF42DFA7668F1024E7D137, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B221A9D82BB34B3CA5299E28B4CD65BD, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:20,345 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791780042_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:20,345 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:20,346 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:20,346 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:20,347 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:20,347 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791780042_0001_00_000001
2016-01-14 17:16:20,347 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791780042_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:20,350 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791780042_0001_00_000001
2016-01-14 17:16:20,350 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:20,351 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791780042_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:20,351 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:20,352 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791780042_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791780042_0001_00_000001, launchTime=1452791780351
2016-01-14 17:16:20,352 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791780042_0001_00_000001
2016-01-14 17:16:20,353 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791780042_0001_1_00_000000_0] to container: [container_1452791780042_0001_00_000001]
2016-01-14 17:16:20,353 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791780042_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:20,354 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791780042_0001_00_000001 given task: attempt_1452791780042_0001_1_00_000000_0
2016-01-14 17:16:20,354 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791780042_0001_1_00_000000_0] started. Is using containerId: [container_1452791780042_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:20,354 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791780042_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791780042_0001_1_00_000000_0
2016-01-14 17:16:20,354 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:20,354 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791780042_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F60C4FE5443841C7BAD85BA367A002A4, taskAttemptId=attempt_1452791780042_0001_1_00_000000_0, startTime=1452791780354, containerId=container_1452791780042_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791780042_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:20,355 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791780042_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:20,355 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791780042_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:20 Starting to run new task attempt: attempt_1452791780042_0001_1_00_000000_0
2016-01-14 17:16:20,356 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791780042_0001_1_00_000000_0
2016-01-14 17:16:20,356 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:20,357 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:20,357 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:20,357 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:20,358 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:20,358 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:20,358 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [15F66FE867924B1D9BC00090E5FD561A/B83A7021DDCA4D2BBF4F120F1AD9C007] every/(1/1) ...atform/-1_'field'_'field', VertexName: F60C4FE5443841C7BAD85BA367A002A4, VertexParallelism: 1, TaskAttemptID:attempt_1452791780042_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B9EAEDE1EDCF42DFA7668F1024E7D137, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B221A9D82BB34B3CA5299E28B4CD65BD, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:20,359 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:16:20,360 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791780042_0001_1_00_000000_0
2016-01-14 17:16:20,360 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:20,361 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:20,360 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791780042_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:20,362 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B9EAEDE1EDCF42DFA7668F1024E7D137, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:20,362 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=B221A9D82BB34B3CA5299E28B4CD65BD, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:20,362 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:20,363 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B9EAEDE1EDCF42DFA7668F1024E7D137
2016-01-14 17:16:20,363 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: B9EAEDE1EDCF42DFA7668F1024E7D137
2016-01-14 17:16:20,362 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:20,364 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:20,364 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: B221A9D82BB34B3CA5299E28B4CD65BD
2016-01-14 17:16:20,368 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:20,369 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: B221A9D82BB34B3CA5299E28B4CD65BD
2016-01-14 17:16:20,369 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: B221A9D82BB34B3CA5299E28B4CD65BD
2016-01-14 17:16:20,370 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:20,373 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:20,374 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: B9EAEDE1EDCF42DFA7668F1024E7D137
2016-01-14 17:16:20,374 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B9EAEDE1EDCF42DFA7668F1024E7D137
2016-01-14 17:16:20,381 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F60C4FE5443841C7BAD85BA367A002A4, ordinal: 0
2016-01-14 17:16:20,382 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F60C4FE5443841C7BAD85BA367A002A4, mem on start (mb), free: 126, total: 281, max: 672
2016-01-14 17:16:20,382 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:20,382 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:20,382 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:20,382 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:20,383 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:20,383 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:20,383 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:20,384 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:20,384 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:20,384 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 17:16:20,385 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B9EAEDE1EDCF42DFA7668F1024E7D137, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:20,385 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, B221A9D82BB34B3CA5299E28B4CD65BD, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:20,385 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:20,386 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B9EAEDE1EDCF42DFA7668F1024E7D137 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:20,386 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:20,387 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:20,386 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B9EAEDE1EDCF42DFA7668F1024E7D137
2016-01-14 17:16:20,387 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:20,388 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B9EAEDE1EDCF42DFA7668F1024E7D137
2016-01-14 17:16:20,388 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:20,389 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791780042_0001_1_00_000000_0
2016-01-14 17:16:20,389 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:20,390 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@460d5f9
2016-01-14 17:16:20,391 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@460d5f9
2016-01-14 17:16:20,391 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:20,392 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:20,394 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:20,394 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: B9EAEDE1EDCF42DFA7668F1024E7D137
2016-01-14 17:16:20,395 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: D92AD3A918444C42ABCD321F13FFF63B
2016-01-14 17:16:20,395 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] D92AD3A918444C42ABCD321F13FFF63B
2016-01-14 17:16:20,396 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@7f2f95e0; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@d18e65c; comparator=cascading.tuple.hadoop.util.TupleComparator@5dbba8bb; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:20,397 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:20,397 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:20,397 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:20,507 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:20,507 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:20,507 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:20,507 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:20,508 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:20,509 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:20,509 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:20,510 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F60C4FE5443841C7BAD85BA367A002A4, all 1 inputs ready in: 00:00:00.001
2016-01-14 17:16:20,514 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791780042_0001_1_00_000000_0
2016-01-14 17:16:20,514 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F60C4FE5443841C7BAD85BA367A002A4, mem on close (mb), free: 124, total: 382, max: 672
2016-01-14 17:16:20,514 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:20,515 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:20,515 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:20,515 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:20,519 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:20 Completed running task attempt: attempt_1452791780042_0001_1_00_000000_0
2016-01-14 17:16:20,524 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791780042_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:20,527 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791780042_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791780042_0001_1_00, eventCount=1
2016-01-14 17:16:20,527 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=400556032, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791780510, Process_Duration=4, Process_End_Time=1452791780514, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:20,528 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791780042_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F60C4FE5443841C7BAD85BA367A002A4, taskAttemptId=attempt_1452791780042_0001_1_00_000000_0, startTime=1452791780354, finishTime=1452791780527, timeTaken=173, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=400556032, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791780510, Process_Duration=4, Process_End_Time=1452791780514, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:20,529 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:20,529 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:20,527 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:20,529 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791780042_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:20,531 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791780042_0001_1_00_000000_0
2016-01-14 17:16:20,531 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:20,532 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791780042_0001_1][Event:TASK_FINISHED]: vertexName=F60C4FE5443841C7BAD85BA367A002A4, taskId=task_1452791780042_0001_1_00_000000, startTime=1452791780354, finishTime=1452791780531, timeTaken=177, status=SUCCEEDED, successfulAttemptID=attempt_1452791780042_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=400556032, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791780510, Process_Duration=4, Process_End_Time=1452791780514, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:20,532 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791780042_0001_00_000001
2016-01-14 17:16:20,532 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:20,532 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791780042_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:20,533 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791780042_0001_1_00 [F60C4FE5443841C7BAD85BA367A002A4] : 1
2016-01-14 17:16:20,533 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791780042_0001_1_00 [F60C4FE5443841C7BAD85BA367A002A4]
2016-01-14 17:16:20,536 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:20,537 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791780042_0001_1][Event:VERTEX_FINISHED]: vertexName=F60C4FE5443841C7BAD85BA367A002A4, vertexId=vertex_1452791780042_0001_1_00, initRequestedTime=1452791780305, initedTime=1452791780325, startRequestedTime=1452791780329, startedTime=1452791780329, finishTime=1452791780535, timeTaken=206, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=400556032, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791780510, Process_Duration=4, Process_End_Time=1452791780514, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791780354, firstTasksToStart=[ task_1452791780042_0001_1_00_000000 ], lastTaskFinishTime=1452791780527, lastTasksToFinish=[ task_1452791780042_0001_1_00_000000 ], minTaskDuration=173, maxTaskDuration=173, avgTaskDuration=173.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791780042_0001_1_00_000000 ], longestDurationTasks=[ task_1452791780042_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:20,537 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791780042_0001_1_00 [F60C4FE5443841C7BAD85BA367A002A4] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:20,537 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791780042_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:20,538 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791780042_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791780042_0001_00_000001, stoppedTime=1452791780538, exitStatus=0
2016-01-14 17:16:20,538 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791780042_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:20,538 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791780042_0001_1_01 [B221A9D82BB34B3CA5299E28B4CD65BD] attempt: attempt_1452791780042_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:20,538 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:20,539 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: B221A9D82BB34B3CA5299E28B4CD65BD
2016-01-14 17:16:20,539 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791780042_0001_1_01 [B221A9D82BB34B3CA5299E28B4CD65BD]
2016-01-14 17:16:20,539 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791780042_0001_1_00 [F60C4FE5443841C7BAD85BA367A002A4] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:20,540 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791780042_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:20,540 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791780042_0001_1][Event:TASK_STARTED]: vertexName=B221A9D82BB34B3CA5299E28B4CD65BD, taskId=task_1452791780042_0001_1_01_000000, scheduledTime=1452791780540, launchTime=1452791780540
2016-01-14 17:16:20,540 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:20,540 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791780042_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:20,541 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791780042_0001_1_01 [B221A9D82BB34B3CA5299E28B4CD65BD]
2016-01-14 17:16:20,541 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791780042_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:20,541 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [15F66FE867924B1D9BC00090E5FD561A/B83A7021DDCA4D2BBF4F120F1AD9C007] every/(1/1) ...atform/-1_'field'_'field', VertexName: B221A9D82BB34B3CA5299E28B4CD65BD, VertexParallelism: 1, TaskAttemptID:attempt_1452791780042_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F60C4FE5443841C7BAD85BA367A002A4, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=3F04FC8D96D947FDAD68E90C70A6D222, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:20,542 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791780042_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:20,542 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:20,543 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791780042_0001_00_000002
2016-01-14 17:16:20,544 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791780042_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:20,545 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791780042_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:20,545 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791780042_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:20,545 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791780042_0001_00_000001, Exiting
2016-01-14 17:16:20,546 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791780042_0001_00_000001
2016-01-14 17:16:20,546 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791780042_0001_00_000001
2016-01-14 17:16:20,548 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791780042_0001_00_000002
2016-01-14 17:16:20,549 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791780042_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:20,549 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 17:16:20,549 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791780042_0001_00_000001 completed successfully
2016-01-14 17:16:20,550 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791780042_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:20,552 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791780042_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:20,552 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:20,552 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791780042_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791780042_0001_00_000002, launchTime=1452791780552
2016-01-14 17:16:20,553 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791780042_0001_00_000002
2016-01-14 17:16:20,554 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791780042_0001_1_01_000000_0] to container: [container_1452791780042_0001_00_000002]
2016-01-14 17:16:20,554 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791780042_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:20,555 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791780042_0001_1_01_000000_0] started. Is using containerId: [container_1452791780042_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:20,555 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791780042_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B221A9D82BB34B3CA5299E28B4CD65BD, taskAttemptId=attempt_1452791780042_0001_1_01_000000_0, startTime=1452791780554, containerId=container_1452791780042_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791780042_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:20,555 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:20,555 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791780042_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:20,556 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791780042_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:20,556 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791780042_0001_00_000002 given task: attempt_1452791780042_0001_1_01_000000_0
2016-01-14 17:16:20,556 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791780042_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791780042_0001_1_01_000000_0
2016-01-14 17:16:20 Starting to run new task attempt: attempt_1452791780042_0001_1_01_000000_0
2016-01-14 17:16:20,557 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791780042_0001_1_01_000000_0
2016-01-14 17:16:20,557 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:20,557 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:20,557 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:20,557 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:20,558 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:20,558 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:20,558 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [15F66FE867924B1D9BC00090E5FD561A/B83A7021DDCA4D2BBF4F120F1AD9C007] every/(1/1) ...atform/-1_'field'_'field', VertexName: B221A9D82BB34B3CA5299E28B4CD65BD, VertexParallelism: 1, TaskAttemptID:attempt_1452791780042_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F60C4FE5443841C7BAD85BA367A002A4, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=3F04FC8D96D947FDAD68E90C70A6D222, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:20,558 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:16:20,559 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791780042_0001_1_01_000000_0
2016-01-14 17:16:20,560 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:20,560 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:20,559 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791780042_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:20,561 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F60C4FE5443841C7BAD85BA367A002A4, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:20,561 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=3F04FC8D96D947FDAD68E90C70A6D222, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:20,561 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:20,562 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:20,564 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F60C4FE5443841C7BAD85BA367A002A4
2016-01-14 17:16:20,568 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:20,562 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:20,569 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 3F04FC8D96D947FDAD68E90C70A6D222
2016-01-14 17:16:20,569 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:20,570 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 3F04FC8D96D947FDAD68E90C70A6D222
2016-01-14 17:16:20,577 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:20,578 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:20,578 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: F60C4FE5443841C7BAD85BA367A002A4
2016-01-14 17:16:20,583 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F60C4FE5443841C7BAD85BA367A002A4
2016-01-14 17:16:20,592 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:20,592 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 3F04FC8D96D947FDAD68E90C70A6D222
2016-01-14 17:16:20,596 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B221A9D82BB34B3CA5299E28B4CD65BD, ordinal: 1
2016-01-14 17:16:20,596 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B221A9D82BB34B3CA5299E28B4CD65BD, mem on start (mb), free: 117, total: 382, max: 672
2016-01-14 17:16:20,597 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:20,597 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:20,597 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:20,597 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:20,597 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:20,597 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:20,598 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:20,599 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:20,599 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:20,600 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:20,600 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 3F04FC8D96D947FDAD68E90C70A6D222, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:20,601 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F60C4FE5443841C7BAD85BA367A002A4, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 17:16:20,601 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:20,602 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F60C4FE5443841C7BAD85BA367A002A4 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:20,602 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F60C4FE5443841C7BAD85BA367A002A4
2016-01-14 17:16:20,603 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:20,604 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:20,604 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:20,605 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:20,605 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: F60C4FE5443841C7BAD85BA367A002A4 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:20,605 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 17:16:20,606 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:20,606 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:20,607 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F60C4FE5443841C7BAD85BA367A002A4
2016-01-14 17:16:20,607 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:20,610 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791780042_0001_1_01_000000_0
2016-01-14 17:16:20,612 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791780042_0001_1_00_000000_0_10009, runDuration: 157218, ]
2016-01-14 17:16:20,617 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791780042_0001_1_00_000000_0_10009] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791780617, TimeTaken=2, Rate=0.14 MB/s
2016-01-14 17:16:20,617 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : F60C4FE5443841C7BAD85BA367A002A4
2016-01-14 17:16:20,617 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:20,617 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:20,617 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [F60C4FE5443841C7BAD85BA367A002A4] #2 in 3ms
2016-01-14 17:16:20,617 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [F60C4FE5443841C7BAD85BA367A002A4] #2, status:false, isInterrupted:false
2016-01-14 17:16:20,618 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:20,618 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: D92AD3A918444C42ABCD321F13FFF63B
2016-01-14 17:16:20,618 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:20,619 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_'field'"], id: 3F04FC8D96D947FDAD68E90C70A6D222
2016-01-14 17:16:20,619 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:20,619 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:20,619 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_'field'"]
2016-01-14 17:16:20,620 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:20,620 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] D92AD3A918444C42ABCD321F13FFF63B, for 1 inputs
2016-01-14 17:16:20,620 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : F60C4FE5443841C7BAD85BA367A002A4
2016-01-14 17:16:20,620 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B221A9D82BB34B3CA5299E28B4CD65BD, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:20,621 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:20,621 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@659b2647; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@2cee336e
2016-01-14 17:16:20,625 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:20,626 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:20,627 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791780042_0001_1_01_000000_0
2016-01-14 17:16:20,627 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791780042_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:20,628 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791780042_0001_1_01_000000_0
2016-01-14 17:16:20,628 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B221A9D82BB34B3CA5299E28B4CD65BD, mem on close (mb), free: 114, total: 382, max: 672
2016-01-14 17:16:20,629 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: F60C4FE5443841C7BAD85BA367A002A4
2016-01-14 17:16:20,629 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:20,629 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:20,630 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:20,630 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:20,630 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:20 Completed running task attempt: attempt_1452791780042_0001_1_01_000000_0
2016-01-14 17:16:20,631 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791780042_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:20,633 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:20,634 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791780042_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B221A9D82BB34B3CA5299E28B4CD65BD, taskAttemptId=attempt_1452791780042_0001_1_01_000000_0, startTime=1452791780554, finishTime=1452791780632, timeTaken=78, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=400556032, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791780621, Process_Duration=7, Process_End_Time=1452791780628, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:20,634 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791780042_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:20,634 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791780042_0001_1_01_000000_0
2016-01-14 17:16:20,636 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791780042_0001_1][Event:TASK_FINISHED]: vertexName=B221A9D82BB34B3CA5299E28B4CD65BD, taskId=task_1452791780042_0001_1_01_000000, startTime=1452791780554, finishTime=1452791780635, timeTaken=81, status=SUCCEEDED, successfulAttemptID=attempt_1452791780042_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=400556032, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791780621, Process_Duration=7, Process_End_Time=1452791780628, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:20,636 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791780042_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:20,637 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=400556032, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791780621, Process_Duration=7, Process_End_Time=1452791780628, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 17:16:20,637 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:20,637 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791780042_0001_1_01 [B221A9D82BB34B3CA5299E28B4CD65BD] : 1
2016-01-14 17:16:20,638 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791780042_0001_1_01 [B221A9D82BB34B3CA5299E28B4CD65BD]
2016-01-14 17:16:20,637 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:20,641 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:20,641 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:20,642 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:20,642 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791780042_0001_1][Event:VERTEX_FINISHED]: vertexName=B221A9D82BB34B3CA5299E28B4CD65BD, vertexId=vertex_1452791780042_0001_1_01, initRequestedTime=1452791780306, initedTime=1452791780323, startRequestedTime=1452791780339, startedTime=1452791780339, finishTime=1452791780639, timeTaken=300, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=400556032, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791780621, Process_Duration=7, Process_End_Time=1452791780628, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452791780554, firstTasksToStart=[ task_1452791780042_0001_1_01_000000 ], lastTaskFinishTime=1452791780632, lastTasksToFinish=[ task_1452791780042_0001_1_01_000000 ], minTaskDuration=78, maxTaskDuration=78, avgTaskDuration=78.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791780042_0001_1_01_000000 ], longestDurationTasks=[ task_1452791780042_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:20,643 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791780042_0001_00_000002
2016-01-14 17:16:20,646 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791780042_0001_1_01 [B221A9D82BB34B3CA5299E28B4CD65BD] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:20,647 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791780042_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:20,647 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791780042_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791780042_0001_00_000002, stoppedTime=1452791780647, exitStatus=0
2016-01-14 17:16:20,647 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791780042_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:20,647 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791780042_0001_1_01 [B221A9D82BB34B3CA5299E28B4CD65BD] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:20,648 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791780042_0001_1
2016-01-14 17:16:20,648 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:20,648 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791780042_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452791780042_0001_1
2016-01-14 17:16:20,648 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 3F04FC8D96D947FDAD68E90C70A6D222 for vertex: vertex_1452791780042_0001_1_01 [B221A9D82BB34B3CA5299E28B4CD65BD]
2016-01-14 17:16:20,657 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791780042_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 17:16:20,657 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791780042_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:20,657 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791780042_0001_00_000002, Exiting
2016-01-14 17:16:20,657 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791780042_0001_00_000002
2016-01-14 17:16:20,657 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791780042_0001_00_000002
2016-01-14 17:16:20,660 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791780042_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:20,660 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791780042_0001_00_000002 completed successfully
2016-01-14 17:16:20,664 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452791780042_0001_1_00 [F60C4FE5443841C7BAD85BA367A002A4]
2016-01-14 17:16:20,668 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791780042_0001_1, queueSize=0
2016-01-14 17:16:20,668 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:20,670 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791780042_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791780042_0001_1, startTime=1452791780304, finishTime=1452791780664, timeTaken=360, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=465, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=801112064, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905583561131, Process_Duration=11, Process_End_Time=2905583561142, Read_Duration=3, Tuples_Read=40, Tuples_Written=40, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 17:16:20,670 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791780042_0001_1 finished with state: SUCCEEDED
2016-01-14 17:16:20,670 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791780042_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:20,670 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452791780042_0001_00_000002
2016-01-14 17:16:20,670 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791780042_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:16:20 Completed Dag: dag_1452791780042_0001_1
2016-01-14 17:16:20,673 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791780042_0001_1, dagState=SUCCEEDED
2016-01-14 17:16:20,673 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791780042_0001_1_post
2016-01-14 17:16:20,673 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:20,674 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:20,681 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...atform/-1_'field'_'field', applicationId=application_1452791780042_0001
2016-01-14 17:16:20,685 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:20,685 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:20,685 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:20,685 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:20,685 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:20,686 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:20,687 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:20,687 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:20,688 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:20,688 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:20,689 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:20,690 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:20,690 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:20,698 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/15F66FE867924B1D9BC00090E5FD561A/B83A7021DDCA4D2BBF4F120F1AD9C007/.tez/application_1452791780042_0001
2016-01-14 17:16:20,699 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.691
2016-01-14 17:16:20,796 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_'field'_'field'
2016-01-14 17:16:20,838 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:20,839 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:20,916 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.038
2016-01-14 17:16:20,932 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.053
2016-01-14 17:16:20,933 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:20,933 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:20,933 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:20,954 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:20,962 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_RESULTS
2016-01-14 17:16:21,020 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:21,021 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:21,021 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_RESULTS"]
2016-01-14 17:16:21,021 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:21,022 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:21,022 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:21,023 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...atform/-1_'field'_RESULTS
2016-01-14 17:16:21,023 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:21,037 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:21,042 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:21,042 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:21,043 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:21,053 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/447A515D066D4CDD96A01DADA5504AE6/DCFA22DEBD06427A99B20335F2C7E10B/.tez/application_1452791781042_0001 doesn't exist and is created
2016-01-14 17:16:21,077 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:21,078 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:21,078 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/447A515D066D4CDD96A01DADA5504AE6/DCFA22DEBD06427A99B20335F2C7E10B/.tez/application_1452791781042_0001
2016-01-14 17:16:21,084 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791781042_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:21,098 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:21,099 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:21,101 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:21,101 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:21,101 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/447A515D066D4CDD96A01DADA5504AE6/DCFA22DEBD06427A99B20335F2C7E10B/.tez/application_1452791781042_0001/localmode-log-dir
2016-01-14 17:16:21,102 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/447A515D066D4CDD96A01DADA5504AE6/DCFA22DEBD06427A99B20335F2C7E10B/.tez/application_1452791781042_0001/localmode-log-dir/history.txt.appattempt_1452791781042_0001_000000, maxErrors=10
2016-01-14 17:16:21,102 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:21,103 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791781042_0001_000000, appSubmitTime=1452791781084, launchTime=1452791781084
2016-01-14 17:16:21,134 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:21,136 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:21,140 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:37538
2016-01-14 17:16:21,140 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:21,141 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:21,147 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:21,151 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791781042_0001_000000, startTime=1452791781151
2016-01-14 17:16:21,151 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:21,151 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:21,178 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:21,178 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:21,178 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...atform/-1_'field'_RESULTS, applicationId=application_1452791781042_0001, dagName=[447A515D066D4CDD96A01DADA5504AE6/DCFA22DEBD06427A99B20335F2C7E10B] every/(1/1) ...atform/-1_'field'_RESULTS
2016-01-14 17:16:21,199 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [447A515D066D4CDD96A01DADA5504AE6/DCFA22DEBD06427A99B20335F2C7E10B] every/(1/1) ...atform/-1_'field'_RESULTS
2016-01-14 17:16:21,201 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791781042_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/447A515D066D4CDD96A01DADA5504AE6/DCFA22DEBD06427A99B20335F2C7E10B/.tez/application_1452791781042_0001/localmode-log-dir/dag_1452791781042_0001_1.dot
2016-01-14 17:16:21,204 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/447A515D066D4CDD96A01DADA5504AE6/DCFA22DEBD06427A99B20335F2C7E10B/.tez/application_1452791781042_0001/localmode-log-dir/dag_1452791781042_0001_1-tez-dag.pb.txt
2016-01-14 17:16:21,233 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791781042_0001_1
2016-01-14 17:16:21,234 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:21,234 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [447A515D066D4CDD96A01DADA5504AE6/DCFA22DEBD06427A99B20335F2C7E10B] every/(1/1) ...atform/-1_'field'_RESULTS
2016-01-14 17:16:21 Running Dag: dag_1452791781042_0001_1
2016-01-14 17:16:21,250 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791781042_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791781042_0001_1, submitTime=1452791781199
2016-01-14 17:16:21,250 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:21,252 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex BE155AF55E154904B4AD851AE216A315
2016-01-14 17:16:21,253 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:21,253 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791781042_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791781042_0001_1, initTime=1452791781251
2016-01-14 17:16:21,253 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:21,254 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791781042_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:21,256 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791781042_0001_1][Event:DAG_STARTED]: dagID=dag_1452791781042_0001_1, startTime=1452791781256
2016-01-14 17:16:21,256 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:21,256 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...atform/-1_'field'_RESULTS, applicationId=application_1452791781042_0001, dagName=[447A515D066D4CDD96A01DADA5504AE6/DCFA22DEBD06427A99B20335F2C7E10B] every/(1/1) ...atform/-1_'field'_RESULTS
2016-01-14 17:16:21,256 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:21,257 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791781042_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:21,257 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: A766D87797F546B5A5F6A92344E2EBDF : {47FAF36A79F34DBDA08EC501F177442C={InputName=47FAF36A79F34DBDA08EC501F177442C}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:21,257 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 47FAF36A79F34DBDA08EC501F177442C, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:21,257 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791781042_0001_1_00 [A766D87797F546B5A5F6A92344E2EBDF]
2016-01-14 17:16:21,257 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791781042_0001_1_00 [A766D87797F546B5A5F6A92344E2EBDF]
2016-01-14 17:16:21,258 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791781042_0001_1_00 [A766D87797F546B5A5F6A92344E2EBDF]
2016-01-14 17:16:21,258 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791781042_0001_1_00 [A766D87797F546B5A5F6A92344E2EBDF]. Starting root input initializers: 1
2016-01-14 17:16:21,258 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791781042_0001_1_00 [A766D87797F546B5A5F6A92344E2EBDF] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:21,259 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791781042_0001_1_01 [BE155AF55E154904B4AD851AE216A315]
2016-01-14 17:16:21,261 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 47FAF36A79F34DBDA08EC501F177442C on vertex vertex_1452791781042_0001_1_00 [A766D87797F546B5A5F6A92344E2EBDF]
2016-01-14 17:16:21,263 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:21,264 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 47FAF36A79F34DBDA08EC501F177442C asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:21,265 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:21,265 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:21,268 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:21,268 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:21,269 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791781042_0001_1_01 [BE155AF55E154904B4AD851AE216A315]
2016-01-14 17:16:21,269 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 47FAF36A79F34DBDA08EC501F177442C on vertex vertex_1452791781042_0001_1_00 [A766D87797F546B5A5F6A92344E2EBDF]
2016-01-14 17:16:21,269 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791781042_0001_1_01 [BE155AF55E154904B4AD851AE216A315]
2016-01-14 17:16:21,271 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791781042_0001_1_01 [BE155AF55E154904B4AD851AE216A315]
2016-01-14 17:16:21,271 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=90DFF64EF0D749658523E986BB332654, vertexId=vertex_1452791781042_0001_1_01 [BE155AF55E154904B4AD851AE216A315], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:21,271 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=90DFF64EF0D749658523E986BB332654, vertexId=vertex_1452791781042_0001_1_01 [BE155AF55E154904B4AD851AE216A315]
2016-01-14 17:16:21,274 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=90DFF64EF0D749658523E986BB332654, vertexName=BE155AF55E154904B4AD851AE216A315, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:21,275 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=90DFF64EF0D749658523E986BB332654, vertexId=vertex_1452791781042_0001_1_01 [BE155AF55E154904B4AD851AE216A315]
2016-01-14 17:16:21,277 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:21,277 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791781042_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BE155AF55E154904B4AD851AE216A315, vertexId=vertex_1452791781042_0001_1_01, initRequestedTime=1452791781259, initedTime=1452791781276, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:21,277 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791781042_0001_1_01 [BE155AF55E154904B4AD851AE216A315] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:21,278 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791781042_0001_1_00 [A766D87797F546B5A5F6A92344E2EBDF] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:21,278 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {47FAF36A79F34DBDA08EC501F177442C=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:21,278 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791781042_0001_1_00 [A766D87797F546B5A5F6A92344E2EBDF] parallelism set to 1
2016-01-14 17:16:21,279 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791781042_0001_1_00 [A766D87797F546B5A5F6A92344E2EBDF] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:21,279 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791781042_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791781042_0001_1_00, eventCount=1
2016-01-14 17:16:21,280 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:21,281 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791781042_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A766D87797F546B5A5F6A92344E2EBDF, vertexId=vertex_1452791781042_0001_1_00, initRequestedTime=1452791781257, initedTime=1452791781280, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:21,281 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791781042_0001_1_00 [A766D87797F546B5A5F6A92344E2EBDF] with distanceFromRoot: 0
2016-01-14 17:16:21,281 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791781042_0001_1_00 [A766D87797F546B5A5F6A92344E2EBDF] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:21,282 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in A766D87797F546B5A5F6A92344E2EBDF
2016-01-14 17:16:21,282 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791781042_0001_1_00 [A766D87797F546B5A5F6A92344E2EBDF]
2016-01-14 17:16:21,283 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:21,283 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791781042_0001, with dag id: dag_1452791781042_0001_1
2016-01-14 17:16:21,284 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791781042_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791781042_0001_1_00, startRequestedTime=1452791781282, startedTime=1452791781282
2016-01-14 17:16:21,284 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791781042_0001_1_00 [A766D87797F546B5A5F6A92344E2EBDF] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:21,287 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:21,288 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791781042_0001_1][Event:TASK_STARTED]: vertexName=A766D87797F546B5A5F6A92344E2EBDF, taskId=task_1452791781042_0001_1_00_000000, scheduledTime=1452791781286, launchTime=1452791781286
2016-01-14 17:16:21,288 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791781042_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:21,288 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791781042_0001_1_00 for vertex: vertex_1452791781042_0001_1_01 [BE155AF55E154904B4AD851AE216A315] numStartedSources: 1 numSources: 1
2016-01-14 17:16:21,291 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791781042_0001_1_01 [BE155AF55E154904B4AD851AE216A315] with distanceFromRoot: 1
2016-01-14 17:16:21,291 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791781042_0001_1_00 [A766D87797F546B5A5F6A92344E2EBDF]
2016-01-14 17:16:21,293 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: A766D87797F546B5A5F6A92344E2EBDF in vertex: BE155AF55E154904B4AD851AE216A315
2016-01-14 17:16:21,294 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: BE155AF55E154904B4AD851AE216A315 with 1 source tasks and 1 pending tasks
2016-01-14 17:16:21,295 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:21,295 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791781042_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791781042_0001_1_01, startRequestedTime=1452791781293, startedTime=1452791781293
2016-01-14 17:16:21,296 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791781042_0001_1_01 [BE155AF55E154904B4AD851AE216A315] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:21,296 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [447A515D066D4CDD96A01DADA5504AE6/DCFA22DEBD06427A99B20335F2C7E10B] every/(1/1) ...atform/-1_'field'_RESULTS, VertexName: A766D87797F546B5A5F6A92344E2EBDF, VertexParallelism: 1, TaskAttemptID:attempt_1452791781042_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=47FAF36A79F34DBDA08EC501F177442C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=BE155AF55E154904B4AD851AE216A315, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:21,297 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791781042_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:21,297 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:21,299 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:21,299 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:21,299 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:21,300 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791781042_0001_00_000001
2016-01-14 17:16:21,300 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791781042_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:21,302 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791781042_0001_00_000001
2016-01-14 17:16:21,302 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:21,303 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791781042_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:21,303 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791781042_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791781042_0001_00_000001, launchTime=1452791781303
2016-01-14 17:16:21,303 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:21,304 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791781042_0001_00_000001
2016-01-14 17:16:21,305 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791781042_0001_1_00_000000_0] to container: [container_1452791781042_0001_00_000001]
2016-01-14 17:16:21,305 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791781042_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:21,305 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791781042_0001_00_000001 given task: attempt_1452791781042_0001_1_00_000000_0
2016-01-14 17:16:21,306 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791781042_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791781042_0001_1_00_000000_0
2016-01-14 17:16:21,306 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791781042_0001_1_00_000000_0] started. Is using containerId: [container_1452791781042_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:21,307 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:21 Starting to run new task attempt: attempt_1452791781042_0001_1_00_000000_0
2016-01-14 17:16:21,307 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791781042_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A766D87797F546B5A5F6A92344E2EBDF, taskAttemptId=attempt_1452791781042_0001_1_00_000000_0, startTime=1452791781306, containerId=container_1452791781042_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791781042_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:21,307 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791781042_0001_1_00_000000_0
2016-01-14 17:16:21,307 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791781042_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:21,307 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:21,307 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:21,308 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:21,308 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:21,308 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791781042_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:21,308 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:21,309 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:21,309 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [447A515D066D4CDD96A01DADA5504AE6/DCFA22DEBD06427A99B20335F2C7E10B] every/(1/1) ...atform/-1_'field'_RESULTS, VertexName: A766D87797F546B5A5F6A92344E2EBDF, VertexParallelism: 1, TaskAttemptID:attempt_1452791781042_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=47FAF36A79F34DBDA08EC501F177442C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=BE155AF55E154904B4AD851AE216A315, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:21,309 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:16:21,310 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791781042_0001_1_00_000000_0
2016-01-14 17:16:21,313 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:21,314 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:21,312 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791781042_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:21,315 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:21,315 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=47FAF36A79F34DBDA08EC501F177442C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:21,316 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:21,316 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 47FAF36A79F34DBDA08EC501F177442C
2016-01-14 17:16:21,316 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 47FAF36A79F34DBDA08EC501F177442C
2016-01-14 17:16:21,316 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=BE155AF55E154904B4AD851AE216A315, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:21,318 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:21,318 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:21,318 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: BE155AF55E154904B4AD851AE216A315
2016-01-14 17:16:21,321 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:21,321 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: BE155AF55E154904B4AD851AE216A315
2016-01-14 17:16:21,322 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:21,322 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: BE155AF55E154904B4AD851AE216A315
2016-01-14 17:16:21,323 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 47FAF36A79F34DBDA08EC501F177442C
2016-01-14 17:16:21,323 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 47FAF36A79F34DBDA08EC501F177442C
2016-01-14 17:16:21,329 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A766D87797F546B5A5F6A92344E2EBDF, ordinal: 0
2016-01-14 17:16:21,330 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A766D87797F546B5A5F6A92344E2EBDF, mem on start (mb), free: 236, total: 490, max: 672
2016-01-14 17:16:21,330 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:21,330 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:21,330 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:21,330 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:21,330 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:21,331 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:21,331 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:21,331 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:21,332 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:21,332 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 17:16:21,333 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 47FAF36A79F34DBDA08EC501F177442C, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:21,333 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, BE155AF55E154904B4AD851AE216A315, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:21,333 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:21,334 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 47FAF36A79F34DBDA08EC501F177442C
2016-01-14 17:16:21,334 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:21,334 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 47FAF36A79F34DBDA08EC501F177442C
2016-01-14 17:16:21,334 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 47FAF36A79F34DBDA08EC501F177442C being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:21,335 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:21,335 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:21,336 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:21,336 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791781042_0001_1_00_000000_0
2016-01-14 17:16:21,336 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:21,337 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@16b1e830
2016-01-14 17:16:21,340 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:21,341 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 47FAF36A79F34DBDA08EC501F177442C
2016-01-14 17:16:21,341 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 49DFAFC40AB6457FA50759A0B21DC339
2016-01-14 17:16:21,341 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@16b1e830
2016-01-14 17:16:21,342 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:21,342 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:21,342 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 49DFAFC40AB6457FA50759A0B21DC339
2016-01-14 17:16:21,343 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@1e871ab4; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@37ea2f14; comparator=cascading.tuple.hadoop.util.TupleComparator@6fb8e6b9; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:21,344 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:21,344 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:21,344 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:21,358 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:21,358 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:21,358 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:21,359 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:21,359 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:21,360 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:21,361 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:21,361 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A766D87797F546B5A5F6A92344E2EBDF, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:21,365 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791781042_0001_1_00_000000_0
2016-01-14 17:16:21,365 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A766D87797F546B5A5F6A92344E2EBDF, mem on close (mb), free: 134, total: 490, max: 672
2016-01-14 17:16:21,365 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:21,365 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:21,366 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:21,366 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:21,370 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:21 Completed running task attempt: attempt_1452791781042_0001_1_00_000000_0
2016-01-14 17:16:21,374 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791781042_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:21,375 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791781042_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791781042_0001_1_00, eventCount=1
2016-01-14 17:16:21,376 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:21,377 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791781361, Process_Duration=4, Process_End_Time=1452791781365, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:21,377 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791781042_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A766D87797F546B5A5F6A92344E2EBDF, taskAttemptId=attempt_1452791781042_0001_1_00_000000_0, startTime=1452791781306, finishTime=1452791781376, timeTaken=70, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791781361, Process_Duration=4, Process_End_Time=1452791781365, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:21,378 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:21,378 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:21,378 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791781042_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:21,378 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791781042_0001_00_000001
2016-01-14 17:16:21,379 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791781042_0001_1_00_000000_0
2016-01-14 17:16:21,379 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:21,380 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:21,380 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791781042_0001_1][Event:TASK_FINISHED]: vertexName=A766D87797F546B5A5F6A92344E2EBDF, taskId=task_1452791781042_0001_1_00_000000, startTime=1452791781306, finishTime=1452791781379, timeTaken=73, status=SUCCEEDED, successfulAttemptID=attempt_1452791781042_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791781361, Process_Duration=4, Process_End_Time=1452791781365, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:21,381 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791781042_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:21,382 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791781042_0001_1_00 [A766D87797F546B5A5F6A92344E2EBDF] : 1
2016-01-14 17:16:21,382 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791781042_0001_1_00 [A766D87797F546B5A5F6A92344E2EBDF]
2016-01-14 17:16:21,383 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:21,384 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791781042_0001_1][Event:VERTEX_FINISHED]: vertexName=A766D87797F546B5A5F6A92344E2EBDF, vertexId=vertex_1452791781042_0001_1_00, initRequestedTime=1452791781257, initedTime=1452791781280, startRequestedTime=1452791781282, startedTime=1452791781282, finishTime=1452791781382, timeTaken=100, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791781361, Process_Duration=4, Process_End_Time=1452791781365, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791781306, firstTasksToStart=[ task_1452791781042_0001_1_00_000000 ], lastTaskFinishTime=1452791781376, lastTasksToFinish=[ task_1452791781042_0001_1_00_000000 ], minTaskDuration=70, maxTaskDuration=70, avgTaskDuration=70.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791781042_0001_1_00_000000 ], longestDurationTasks=[ task_1452791781042_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:21,385 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791781042_0001_1_00 [A766D87797F546B5A5F6A92344E2EBDF] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:21,385 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791781042_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:21,385 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791781042_0001_1_01 [BE155AF55E154904B4AD851AE216A315] attempt: attempt_1452791781042_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:21,386 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: BE155AF55E154904B4AD851AE216A315
2016-01-14 17:16:21,386 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791781042_0001_1_01 [BE155AF55E154904B4AD851AE216A315]
2016-01-14 17:16:21,387 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:21,387 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791781042_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791781042_0001_00_000001, stoppedTime=1452791781387, exitStatus=0
2016-01-14 17:16:21,388 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791781042_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:21,388 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791781042_0001_1_00 [A766D87797F546B5A5F6A92344E2EBDF] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:21,389 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791781042_0001_1][Event:TASK_STARTED]: vertexName=BE155AF55E154904B4AD851AE216A315, taskId=task_1452791781042_0001_1_01_000000, scheduledTime=1452791781389, launchTime=1452791781389
2016-01-14 17:16:21,389 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:21,390 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791781042_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:21,390 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791781042_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:21,390 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791781042_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:21,390 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791781042_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:21,391 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791781042_0001_00_000001, Exiting
2016-01-14 17:16:21,391 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791781042_0001_00_000001
2016-01-14 17:16:21,391 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791781042_0001_00_000001
2016-01-14 17:16:21,391 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791781042_0001_1_01 [BE155AF55E154904B4AD851AE216A315]
2016-01-14 17:16:21,392 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791781042_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:21,393 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [447A515D066D4CDD96A01DADA5504AE6/DCFA22DEBD06427A99B20335F2C7E10B] every/(1/1) ...atform/-1_'field'_RESULTS, VertexName: BE155AF55E154904B4AD851AE216A315, VertexParallelism: 1, TaskAttemptID:attempt_1452791781042_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A766D87797F546B5A5F6A92344E2EBDF, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=90DFF64EF0D749658523E986BB332654, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:21,393 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791781042_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:21,393 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:21,394 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791781042_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:21,394 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791781042_0001_00_000001 completed successfully
2016-01-14 17:16:21,395 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791781042_0001_00_000002
2016-01-14 17:16:21,395 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791781042_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:21,395 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791781042_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:21,397 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791781042_0001_00_000002
2016-01-14 17:16:21,398 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 17:16:21,401 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791781042_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:21,401 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:21,401 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791781042_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791781042_0001_00_000002, launchTime=1452791781400
2016-01-14 17:16:21,403 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791781042_0001_00_000002
2016-01-14 17:16:21,403 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791781042_0001_1_01_000000_0] to container: [container_1452791781042_0001_00_000002]
2016-01-14 17:16:21,404 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791781042_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:21,406 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791781042_0001_1_01_000000_0] started. Is using containerId: [container_1452791781042_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:21,406 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:21,407 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791781042_0001_00_000002 given task: attempt_1452791781042_0001_1_01_000000_0
2016-01-14 17:16:21,407 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791781042_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791781042_0001_1_01_000000_0
2016-01-14 17:16:21,407 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791781042_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BE155AF55E154904B4AD851AE216A315, taskAttemptId=attempt_1452791781042_0001_1_01_000000_0, startTime=1452791781405, containerId=container_1452791781042_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791781042_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:21,407 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791781042_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:21 Starting to run new task attempt: attempt_1452791781042_0001_1_01_000000_0
2016-01-14 17:16:21,407 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791781042_0001_1_01_000000_0
2016-01-14 17:16:21,408 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:21,408 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:21,408 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:21,409 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:21,409 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:21,409 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:21,410 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [447A515D066D4CDD96A01DADA5504AE6/DCFA22DEBD06427A99B20335F2C7E10B] every/(1/1) ...atform/-1_'field'_RESULTS, VertexName: BE155AF55E154904B4AD851AE216A315, VertexParallelism: 1, TaskAttemptID:attempt_1452791781042_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A766D87797F546B5A5F6A92344E2EBDF, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=90DFF64EF0D749658523E986BB332654, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:21,410 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:16:21,410 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791781042_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:21,411 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791781042_0001_1_01_000000_0
2016-01-14 17:16:21,412 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:21,412 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791781042_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:21,412 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:21,413 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=A766D87797F546B5A5F6A92344E2EBDF, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:21,413 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=90DFF64EF0D749658523E986BB332654, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:21,414 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:21,414 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: A766D87797F546B5A5F6A92344E2EBDF
2016-01-14 17:16:21,413 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:21,415 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:21,415 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 90DFF64EF0D749658523E986BB332654
2016-01-14 17:16:21,415 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:21,415 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 90DFF64EF0D749658523E986BB332654
2016-01-14 17:16:21,417 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:21,417 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:21,417 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: A766D87797F546B5A5F6A92344E2EBDF
2016-01-14 17:16:21,418 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: A766D87797F546B5A5F6A92344E2EBDF
2016-01-14 17:16:21,423 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:21,431 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:21,431 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 90DFF64EF0D749658523E986BB332654
2016-01-14 17:16:21,433 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BE155AF55E154904B4AD851AE216A315, ordinal: 1
2016-01-14 17:16:21,433 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BE155AF55E154904B4AD851AE216A315, mem on start (mb), free: 126, total: 490, max: 672
2016-01-14 17:16:21,434 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:21,434 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:21,434 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:21,434 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:21,434 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:21,435 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:21,435 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:21,435 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:21,436 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:21,436 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:21,436 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 90DFF64EF0D749658523E986BB332654, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:21,436 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, A766D87797F546B5A5F6A92344E2EBDF, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 17:16:21,437 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:21,437 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: A766D87797F546B5A5F6A92344E2EBDF
2016-01-14 17:16:21,437 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: A766D87797F546B5A5F6A92344E2EBDF being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:21,437 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:21,438 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:21,439 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:21,439 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: A766D87797F546B5A5F6A92344E2EBDF with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:21,439 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:21,439 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 17:16:21,440 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:21,440 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:21,445 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: A766D87797F546B5A5F6A92344E2EBDF
2016-01-14 17:16:21,445 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:21,446 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791781042_0001_1_01_000000_0
2016-01-14 17:16:21,446 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791781042_0001_1_00_000000_0_10015, runDuration: 54018, ]
2016-01-14 17:16:21,448 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791781042_0001_1_00_000000_0_10015] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791781448, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 17:16:21,448 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : A766D87797F546B5A5F6A92344E2EBDF
2016-01-14 17:16:21,448 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:21,448 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [A766D87797F546B5A5F6A92344E2EBDF] #3 in 1ms
2016-01-14 17:16:21,449 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [A766D87797F546B5A5F6A92344E2EBDF] #3, status:false, isInterrupted:false
2016-01-14 17:16:21,450 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:21,451 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:21,451 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:21,452 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:21,452 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:21,453 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:21,453 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 49DFAFC40AB6457FA50759A0B21DC339
2016-01-14 17:16:21,453 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : A766D87797F546B5A5F6A92344E2EBDF
2016-01-14 17:16:21,453 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_RESULTS"], id: 90DFF64EF0D749658523E986BB332654
2016-01-14 17:16:21,454 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:21,454 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_RESULTS"]
2016-01-14 17:16:21,455 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 49DFAFC40AB6457FA50759A0B21DC339, for 1 inputs
2016-01-14 17:16:21,455 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BE155AF55E154904B4AD851AE216A315, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:21,456 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@61ea5137; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@5e7a2419
2016-01-14 17:16:21,460 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:21,461 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:21,461 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791781042_0001_1_01_000000_0
2016-01-14 17:16:21,462 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791781042_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:21,465 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791781042_0001_1_01_000000_0
2016-01-14 17:16:21,465 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BE155AF55E154904B4AD851AE216A315, mem on close (mb), free: 122, total: 490, max: 672
2016-01-14 17:16:21,465 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: A766D87797F546B5A5F6A92344E2EBDF
2016-01-14 17:16:21,466 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:21,466 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:21,466 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:21,467 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:21,467 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:21 Completed running task attempt: attempt_1452791781042_0001_1_01_000000_0
2016-01-14 17:16:21,468 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791781042_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:21,470 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791781455, Process_Duration=10, Process_End_Time=1452791781465, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 17:16:21,470 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791781042_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BE155AF55E154904B4AD851AE216A315, taskAttemptId=attempt_1452791781042_0001_1_01_000000_0, startTime=1452791781405, finishTime=1452791781469, timeTaken=64, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791781455, Process_Duration=10, Process_End_Time=1452791781465, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:21,470 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:21,471 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791781042_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:21,471 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791781042_0001_1_01_000000_0
2016-01-14 17:16:21,471 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:21,470 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:21,472 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:21,472 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:21,473 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791781042_0001_00_000002
2016-01-14 17:16:21,472 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791781042_0001_1][Event:TASK_FINISHED]: vertexName=BE155AF55E154904B4AD851AE216A315, taskId=task_1452791781042_0001_1_01_000000, startTime=1452791781405, finishTime=1452791781471, timeTaken=66, status=SUCCEEDED, successfulAttemptID=attempt_1452791781042_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791781455, Process_Duration=10, Process_End_Time=1452791781465, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:21,473 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791781042_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:21,474 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791781042_0001_1_01 [BE155AF55E154904B4AD851AE216A315] : 1
2016-01-14 17:16:21,474 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791781042_0001_1_01 [BE155AF55E154904B4AD851AE216A315]
2016-01-14 17:16:21,475 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:21,476 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791781042_0001_1][Event:VERTEX_FINISHED]: vertexName=BE155AF55E154904B4AD851AE216A315, vertexId=vertex_1452791781042_0001_1_01, initRequestedTime=1452791781259, initedTime=1452791781276, startRequestedTime=1452791781293, startedTime=1452791781293, finishTime=1452791781474, timeTaken=181, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791781455, Process_Duration=10, Process_End_Time=1452791781465, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452791781405, firstTasksToStart=[ task_1452791781042_0001_1_01_000000 ], lastTaskFinishTime=1452791781469, lastTasksToFinish=[ task_1452791781042_0001_1_01_000000 ], minTaskDuration=64, maxTaskDuration=64, avgTaskDuration=64.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791781042_0001_1_01_000000 ], longestDurationTasks=[ task_1452791781042_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:21,476 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791781042_0001_1_01 [BE155AF55E154904B4AD851AE216A315] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:21,477 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791781042_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:21,478 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791781042_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791781042_0001_00_000002, stoppedTime=1452791781478, exitStatus=0
2016-01-14 17:16:21,478 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791781042_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:21,478 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791781042_0001_1_01 [BE155AF55E154904B4AD851AE216A315] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:21,478 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791781042_0001_1
2016-01-14 17:16:21,479 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791781042_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452791781042_0001_1
2016-01-14 17:16:21,479 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 90DFF64EF0D749658523E986BB332654 for vertex: vertex_1452791781042_0001_1_01 [BE155AF55E154904B4AD851AE216A315]
2016-01-14 17:16:21,478 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:21,483 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791781042_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 17:16:21,483 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791781042_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:21,484 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791781042_0001_00_000002, Exiting
2016-01-14 17:16:21,484 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791781042_0001_00_000002
2016-01-14 17:16:21,484 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791781042_0001_00_000002
2016-01-14 17:16:21,485 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791781042_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:21,485 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791781042_0001_00_000002 completed successfully
2016-01-14 17:16:21,491 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452791781042_0001_1_00 [A766D87797F546B5A5F6A92344E2EBDF]
2016-01-14 17:16:21,494 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791781042_0001_1, queueSize=0
2016-01-14 17:16:21,494 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:21,494 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791781042_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791781042_0001_1, startTime=1452791781256, finishTime=1452791781491, timeTaken=235, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=465, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1028653056, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905583562816, Process_Duration=14, Process_End_Time=2905583562830, Read_Duration=5, Tuples_Read=40, Tuples_Written=40, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 17:16:21,495 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791781042_0001_1 finished with state: SUCCEEDED
2016-01-14 17:16:21,495 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791781042_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:21,495 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452791781042_0001_00_000002
2016-01-14 17:16:21,495 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791781042_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:16:21 Completed Dag: dag_1452791781042_0001_1
2016-01-14 17:16:21,498 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791781042_0001_1, dagState=SUCCEEDED
2016-01-14 17:16:21,498 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791781042_0001_1_post
2016-01-14 17:16:21,499 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:21,499 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:21,509 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...atform/-1_'field'_RESULTS, applicationId=application_1452791781042_0001
2016-01-14 17:16:21,511 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:21,511 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:21,511 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:21,511 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:21,511 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:21,512 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:21,512 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:21,514 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:21,514 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:21,514 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:21,515 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:21,515 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:21,516 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:21,522 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/447A515D066D4CDD96A01DADA5504AE6/DCFA22DEBD06427A99B20335F2C7E10B/.tez/application_1452791781042_0001
2016-01-14 17:16:21,522 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.502
2016-01-14 17:16:21,640 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_'field'_RESULTS
2016-01-14 17:16:21,676 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:21,676 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:21,743 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.028
2016-01-14 17:16:21,767 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.052
2016-01-14 17:16:21,768 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:21,768 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:21,769 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:21,793 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:21,803 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_ALL
2016-01-14 17:16:21,880 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:21,880 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:21,880 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_ALL"]
2016-01-14 17:16:21,881 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:21,881 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:21,881 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:21,883 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...lyplatform/-1_'field'_ALL
2016-01-14 17:16:21,883 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:21,902 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:21,907 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:21,907 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:21,907 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:21,917 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E9044769E91741B89DDA6C94FE9D4DFA/AA31AD92EBAC41A5A1398BCF6AD382E1/.tez/application_1452791781907_0001 doesn't exist and is created
2016-01-14 17:16:21,936 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:21,936 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:21,937 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E9044769E91741B89DDA6C94FE9D4DFA/AA31AD92EBAC41A5A1398BCF6AD382E1/.tez/application_1452791781907_0001
2016-01-14 17:16:21,942 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791781907_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:21,960 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:21,961 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:21,963 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:21,964 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:21,964 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E9044769E91741B89DDA6C94FE9D4DFA/AA31AD92EBAC41A5A1398BCF6AD382E1/.tez/application_1452791781907_0001/localmode-log-dir
2016-01-14 17:16:21,964 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E9044769E91741B89DDA6C94FE9D4DFA/AA31AD92EBAC41A5A1398BCF6AD382E1/.tez/application_1452791781907_0001/localmode-log-dir/history.txt.appattempt_1452791781907_0001_000000, maxErrors=10
2016-01-14 17:16:21,964 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:21,965 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791781907_0001_000000, appSubmitTime=1452791781941, launchTime=1452791781941
2016-01-14 17:16:21,997 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:21,998 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:53452
2016-01-14 17:16:22,000 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:22,003 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:22,004 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:22,005 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:22,008 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791781907_0001_000000, startTime=1452791782008
2016-01-14 17:16:22,008 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:22,008 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:22,037 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:22,037 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:22,037 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...lyplatform/-1_'field'_ALL, applicationId=application_1452791781907_0001, dagName=[E9044769E91741B89DDA6C94FE9D4DFA/AA31AD92EBAC41A5A1398BCF6AD382E1] every/(1/1) ...lyplatform/-1_'field'_ALL
2016-01-14 17:16:22,057 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [E9044769E91741B89DDA6C94FE9D4DFA/AA31AD92EBAC41A5A1398BCF6AD382E1] every/(1/1) ...lyplatform/-1_'field'_ALL
2016-01-14 17:16:22,059 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791781907_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E9044769E91741B89DDA6C94FE9D4DFA/AA31AD92EBAC41A5A1398BCF6AD382E1/.tez/application_1452791781907_0001/localmode-log-dir/dag_1452791781907_0001_1.dot
2016-01-14 17:16:22,061 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E9044769E91741B89DDA6C94FE9D4DFA/AA31AD92EBAC41A5A1398BCF6AD382E1/.tez/application_1452791781907_0001/localmode-log-dir/dag_1452791781907_0001_1-tez-dag.pb.txt
2016-01-14 17:16:22,093 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791781907_0001_1
2016-01-14 17:16:22,093 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:22,093 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [E9044769E91741B89DDA6C94FE9D4DFA/AA31AD92EBAC41A5A1398BCF6AD382E1] every/(1/1) ...lyplatform/-1_'field'_ALL
2016-01-14 17:16:22 Running Dag: dag_1452791781907_0001_1
2016-01-14 17:16:22,107 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:22,108 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791781907_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791781907_0001_1, submitTime=1452791782058
2016-01-14 17:16:22,109 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex B99E522F96164E14BBF172676F2D083D
2016-01-14 17:16:22,109 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:22,110 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791781907_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791781907_0001_1, initTime=1452791782108
2016-01-14 17:16:22,110 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:22,110 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791781907_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:22,111 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791781907_0001_1][Event:DAG_STARTED]: dagID=dag_1452791781907_0001_1, startTime=1452791782111
2016-01-14 17:16:22,111 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:22,111 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:22,112 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791781907_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:22,112 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D42A4A5819E64448AC3DEE386CBA863E : {0DB8695E0A9A452D9AF4190B3E3F8AA2={InputName=0DB8695E0A9A452D9AF4190B3E3F8AA2}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:22,112 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 0DB8695E0A9A452D9AF4190B3E3F8AA2, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:22,112 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791781907_0001_1_00 [D42A4A5819E64448AC3DEE386CBA863E]
2016-01-14 17:16:22,112 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791781907_0001_1_00 [D42A4A5819E64448AC3DEE386CBA863E]
2016-01-14 17:16:22,112 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791781907_0001_1_00 [D42A4A5819E64448AC3DEE386CBA863E]
2016-01-14 17:16:22,113 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791781907_0001_1_00 [D42A4A5819E64448AC3DEE386CBA863E]. Starting root input initializers: 1
2016-01-14 17:16:22,113 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...lyplatform/-1_'field'_ALL, applicationId=application_1452791781907_0001, dagName=[E9044769E91741B89DDA6C94FE9D4DFA/AA31AD92EBAC41A5A1398BCF6AD382E1] every/(1/1) ...lyplatform/-1_'field'_ALL
2016-01-14 17:16:22,114 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791781907_0001_1_00 [D42A4A5819E64448AC3DEE386CBA863E] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:22,114 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791781907_0001_1_01 [B99E522F96164E14BBF172676F2D083D]
2016-01-14 17:16:22,118 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 0DB8695E0A9A452D9AF4190B3E3F8AA2 on vertex vertex_1452791781907_0001_1_00 [D42A4A5819E64448AC3DEE386CBA863E]
2016-01-14 17:16:22,119 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:22,119 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791781907_0001_1_01 [B99E522F96164E14BBF172676F2D083D]
2016-01-14 17:16:22,119 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:22,120 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 0DB8695E0A9A452D9AF4190B3E3F8AA2 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:22,120 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791781907_0001_1_01 [B99E522F96164E14BBF172676F2D083D]
2016-01-14 17:16:22,121 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791781907_0001_1_01 [B99E522F96164E14BBF172676F2D083D]
2016-01-14 17:16:22,121 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=447C12C2E092494CBFEFC5A294C6CD85, vertexId=vertex_1452791781907_0001_1_01 [B99E522F96164E14BBF172676F2D083D], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:22,122 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=447C12C2E092494CBFEFC5A294C6CD85, vertexId=vertex_1452791781907_0001_1_01 [B99E522F96164E14BBF172676F2D083D]
2016-01-14 17:16:22,121 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:22,123 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:22,125 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=447C12C2E092494CBFEFC5A294C6CD85, vertexName=B99E522F96164E14BBF172676F2D083D, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:22,125 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:22,126 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=447C12C2E092494CBFEFC5A294C6CD85, vertexId=vertex_1452791781907_0001_1_01 [B99E522F96164E14BBF172676F2D083D]
2016-01-14 17:16:22,126 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 0DB8695E0A9A452D9AF4190B3E3F8AA2 on vertex vertex_1452791781907_0001_1_00 [D42A4A5819E64448AC3DEE386CBA863E]
2016-01-14 17:16:22,129 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:22,129 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791781907_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B99E522F96164E14BBF172676F2D083D, vertexId=vertex_1452791781907_0001_1_01, initRequestedTime=1452791782114, initedTime=1452791782128, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:22,130 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791781907_0001_1_01 [B99E522F96164E14BBF172676F2D083D] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:22,130 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791781907_0001_1_00 [D42A4A5819E64448AC3DEE386CBA863E] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:22,131 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {0DB8695E0A9A452D9AF4190B3E3F8AA2=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:22,131 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791781907_0001_1_00 [D42A4A5819E64448AC3DEE386CBA863E] parallelism set to 1
2016-01-14 17:16:22,132 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791781907_0001_1_00 [D42A4A5819E64448AC3DEE386CBA863E] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:22,133 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791781907_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791781907_0001_1_00, eventCount=1
2016-01-14 17:16:22,134 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:22,134 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791781907_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D42A4A5819E64448AC3DEE386CBA863E, vertexId=vertex_1452791781907_0001_1_00, initRequestedTime=1452791782112, initedTime=1452791782133, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:22,135 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791781907_0001_1_00 [D42A4A5819E64448AC3DEE386CBA863E] with distanceFromRoot: 0
2016-01-14 17:16:22,135 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791781907_0001_1_00 [D42A4A5819E64448AC3DEE386CBA863E] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:22,136 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791781907_0001, with dag id: dag_1452791781907_0001_1
2016-01-14 17:16:22,136 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D42A4A5819E64448AC3DEE386CBA863E
2016-01-14 17:16:22,136 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791781907_0001_1_00 [D42A4A5819E64448AC3DEE386CBA863E]
2016-01-14 17:16:22,137 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:22,137 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791781907_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791781907_0001_1_00, startRequestedTime=1452791782136, startedTime=1452791782136
2016-01-14 17:16:22,137 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791781907_0001_1_00 [D42A4A5819E64448AC3DEE386CBA863E] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:22,138 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:22,139 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791781907_0001_1][Event:TASK_STARTED]: vertexName=D42A4A5819E64448AC3DEE386CBA863E, taskId=task_1452791781907_0001_1_00_000000, scheduledTime=1452791782138, launchTime=1452791782138
2016-01-14 17:16:22,140 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791781907_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:22,141 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791781907_0001_1_00 for vertex: vertex_1452791781907_0001_1_01 [B99E522F96164E14BBF172676F2D083D] numStartedSources: 1 numSources: 1
2016-01-14 17:16:22,141 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791781907_0001_1_01 [B99E522F96164E14BBF172676F2D083D] with distanceFromRoot: 1
2016-01-14 17:16:22,141 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791781907_0001_1_00 [D42A4A5819E64448AC3DEE386CBA863E]
2016-01-14 17:16:22,142 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: D42A4A5819E64448AC3DEE386CBA863E in vertex: B99E522F96164E14BBF172676F2D083D
2016-01-14 17:16:22,142 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: B99E522F96164E14BBF172676F2D083D with 1 source tasks and 1 pending tasks
2016-01-14 17:16:22,142 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:22,143 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791781907_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791781907_0001_1_01, startRequestedTime=1452791782141, startedTime=1452791782141
2016-01-14 17:16:22,143 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791781907_0001_1_01 [B99E522F96164E14BBF172676F2D083D] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:22,143 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [E9044769E91741B89DDA6C94FE9D4DFA/AA31AD92EBAC41A5A1398BCF6AD382E1] every/(1/1) ...lyplatform/-1_'field'_ALL, VertexName: D42A4A5819E64448AC3DEE386CBA863E, VertexParallelism: 1, TaskAttemptID:attempt_1452791781907_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=0DB8695E0A9A452D9AF4190B3E3F8AA2, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B99E522F96164E14BBF172676F2D083D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:22,144 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791781907_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:22,144 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:22,146 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:22,146 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:22,147 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:22,148 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791781907_0001_00_000001
2016-01-14 17:16:22,148 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791781907_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:22,150 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791781907_0001_00_000001
2016-01-14 17:16:22,151 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:22,151 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791781907_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:22,152 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:22,152 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791781907_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791781907_0001_00_000001, launchTime=1452791782152
2016-01-14 17:16:22,153 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791781907_0001_00_000001
2016-01-14 17:16:22,154 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791781907_0001_1_00_000000_0] to container: [container_1452791781907_0001_00_000001]
2016-01-14 17:16:22,154 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791781907_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:22,154 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791781907_0001_00_000001 given task: attempt_1452791781907_0001_1_00_000000_0
2016-01-14 17:16:22,154 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791781907_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791781907_0001_1_00_000000_0
2016-01-14 17:16:22,155 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791781907_0001_1_00_000000_0] started. Is using containerId: [container_1452791781907_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:22,155 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:22,156 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791781907_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D42A4A5819E64448AC3DEE386CBA863E, taskAttemptId=attempt_1452791781907_0001_1_00_000000_0, startTime=1452791782154, containerId=container_1452791781907_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791781907_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:22,156 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791781907_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:22,156 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791781907_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:22 Starting to run new task attempt: attempt_1452791781907_0001_1_00_000000_0
2016-01-14 17:16:22,157 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791781907_0001_1_00_000000_0
2016-01-14 17:16:22,157 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:22,157 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:22,157 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:22,157 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:22,157 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:22,157 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:22,158 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [E9044769E91741B89DDA6C94FE9D4DFA/AA31AD92EBAC41A5A1398BCF6AD382E1] every/(1/1) ...lyplatform/-1_'field'_ALL, VertexName: D42A4A5819E64448AC3DEE386CBA863E, VertexParallelism: 1, TaskAttemptID:attempt_1452791781907_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=0DB8695E0A9A452D9AF4190B3E3F8AA2, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B99E522F96164E14BBF172676F2D083D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:22,158 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:16:22,158 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791781907_0001_1_00_000000_0
2016-01-14 17:16:22,158 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:22,161 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:22,161 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=0DB8695E0A9A452D9AF4190B3E3F8AA2, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:22,163 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:22,163 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 0DB8695E0A9A452D9AF4190B3E3F8AA2
2016-01-14 17:16:22,164 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 0DB8695E0A9A452D9AF4190B3E3F8AA2
2016-01-14 17:16:22,162 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791781907_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:22,163 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=B99E522F96164E14BBF172676F2D083D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:22,163 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:22,167 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:22,168 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:22,168 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: B99E522F96164E14BBF172676F2D083D
2016-01-14 17:16:22,169 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 0DB8695E0A9A452D9AF4190B3E3F8AA2
2016-01-14 17:16:22,169 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 0DB8695E0A9A452D9AF4190B3E3F8AA2
2016-01-14 17:16:22,170 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:22,170 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:22,170 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: B99E522F96164E14BBF172676F2D083D
2016-01-14 17:16:22,171 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: B99E522F96164E14BBF172676F2D083D
2016-01-14 17:16:22,178 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D42A4A5819E64448AC3DEE386CBA863E, ordinal: 0
2016-01-14 17:16:22,179 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D42A4A5819E64448AC3DEE386CBA863E, mem on start (mb), free: 232, total: 483, max: 672
2016-01-14 17:16:22,179 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:22,179 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:22,179 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:22,180 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:22,180 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:22,180 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:22,181 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:22,181 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:22,181 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:22,182 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 17:16:22,182 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 0DB8695E0A9A452D9AF4190B3E3F8AA2, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:22,182 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, B99E522F96164E14BBF172676F2D083D, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:22,182 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:22,183 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 0DB8695E0A9A452D9AF4190B3E3F8AA2
2016-01-14 17:16:22,183 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:22,183 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 0DB8695E0A9A452D9AF4190B3E3F8AA2
2016-01-14 17:16:22,182 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 0DB8695E0A9A452D9AF4190B3E3F8AA2 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:22,184 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:22,184 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:22,184 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:22,185 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791781907_0001_1_00_000000_0
2016-01-14 17:16:22,185 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:22,187 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@5b85ec5d
2016-01-14 17:16:22,188 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:22,189 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 0DB8695E0A9A452D9AF4190B3E3F8AA2
2016-01-14 17:16:22,189 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@5b85ec5d
2016-01-14 17:16:22,189 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:22,189 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:22,189 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 183062C428334711978011C737EA018B
2016-01-14 17:16:22,190 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 183062C428334711978011C737EA018B
2016-01-14 17:16:22,193 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@6d7329a1; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@23b2412a; comparator=cascading.tuple.hadoop.util.TupleComparator@bd43eb5; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:22,194 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:22,194 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:22,195 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:22,209 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:22,210 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:22,210 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:22,210 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:22,211 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:22,211 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:22,212 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:22,212 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D42A4A5819E64448AC3DEE386CBA863E, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:22,215 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791781907_0001_1_00_000000_0
2016-01-14 17:16:22,216 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D42A4A5819E64448AC3DEE386CBA863E, mem on close (mb), free: 130, total: 483, max: 672
2016-01-14 17:16:22,216 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:22,216 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:22,216 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:22,217 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:22,222 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:22 Completed running task attempt: attempt_1452791781907_0001_1_00_000000_0
2016-01-14 17:16:22,227 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791781907_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:22,228 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791781907_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791781907_0001_1_00, eventCount=1
2016-01-14 17:16:22,229 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:22,229 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791781907_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D42A4A5819E64448AC3DEE386CBA863E, taskAttemptId=attempt_1452791781907_0001_1_00_000000_0, startTime=1452791782154, finishTime=1452791782229, timeTaken=75, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=506986496, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791782212, Process_Duration=3, Process_End_Time=1452791782215, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:22,230 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791781907_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:22,230 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791781907_0001_1_00_000000_0
2016-01-14 17:16:22,230 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:22,231 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:22,231 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791781907_0001_1][Event:TASK_FINISHED]: vertexName=D42A4A5819E64448AC3DEE386CBA863E, taskId=task_1452791781907_0001_1_00_000000, startTime=1452791782154, finishTime=1452791782230, timeTaken=76, status=SUCCEEDED, successfulAttemptID=attempt_1452791781907_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=506986496, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791782212, Process_Duration=3, Process_End_Time=1452791782215, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:22,231 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791781907_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:22,232 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791781907_0001_1_00 [D42A4A5819E64448AC3DEE386CBA863E] : 1
2016-01-14 17:16:22,232 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791781907_0001_1_00 [D42A4A5819E64448AC3DEE386CBA863E]
2016-01-14 17:16:22,233 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:22,233 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791781907_0001_1][Event:VERTEX_FINISHED]: vertexName=D42A4A5819E64448AC3DEE386CBA863E, vertexId=vertex_1452791781907_0001_1_00, initRequestedTime=1452791782112, initedTime=1452791782133, startRequestedTime=1452791782136, startedTime=1452791782136, finishTime=1452791782232, timeTaken=96, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=506986496, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791782212, Process_Duration=3, Process_End_Time=1452791782215, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791782154, firstTasksToStart=[ task_1452791781907_0001_1_00_000000 ], lastTaskFinishTime=1452791782229, lastTasksToFinish=[ task_1452791781907_0001_1_00_000000 ], minTaskDuration=75, maxTaskDuration=75, avgTaskDuration=75.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791781907_0001_1_00_000000 ], longestDurationTasks=[ task_1452791781907_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:22,233 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791781907_0001_1_00 [D42A4A5819E64448AC3DEE386CBA863E] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:22,234 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791781907_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:22,234 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791781907_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791781907_0001_00_000001, stoppedTime=1452791782234, exitStatus=0
2016-01-14 17:16:22,234 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=506986496, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791782212, Process_Duration=3, Process_End_Time=1452791782215, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:22,234 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:22,235 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:22,236 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:22,236 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791781907_0001_00_000001
2016-01-14 17:16:22,235 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791781907_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:22,236 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791781907_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:22,237 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791781907_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:22,237 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791781907_0001_00_000001, Exiting
2016-01-14 17:16:22,237 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791781907_0001_00_000001
2016-01-14 17:16:22,237 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791781907_0001_00_000001
2016-01-14 17:16:22,236 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791781907_0001_1_01 [B99E522F96164E14BBF172676F2D083D] attempt: attempt_1452791781907_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:22,239 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: B99E522F96164E14BBF172676F2D083D
2016-01-14 17:16:22,240 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791781907_0001_1_01 [B99E522F96164E14BBF172676F2D083D]
2016-01-14 17:16:22,240 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791781907_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:22,240 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791781907_0001_00_000001 completed successfully
2016-01-14 17:16:22,241 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791781907_0001_1_00 [D42A4A5819E64448AC3DEE386CBA863E] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:22,241 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452791781907_0001_00_000001
2016-01-14 17:16:22,241 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:22,241 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791781907_0001_1][Event:TASK_STARTED]: vertexName=B99E522F96164E14BBF172676F2D083D, taskId=task_1452791781907_0001_1_01_000000, scheduledTime=1452791782241, launchTime=1452791782241
2016-01-14 17:16:22,242 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791781907_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:22,242 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791781907_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:16:22,242 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791781907_0001_1_01 [B99E522F96164E14BBF172676F2D083D]
2016-01-14 17:16:22,243 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [E9044769E91741B89DDA6C94FE9D4DFA/AA31AD92EBAC41A5A1398BCF6AD382E1] every/(1/1) ...lyplatform/-1_'field'_ALL, VertexName: B99E522F96164E14BBF172676F2D083D, VertexParallelism: 1, TaskAttemptID:attempt_1452791781907_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D42A4A5819E64448AC3DEE386CBA863E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=447C12C2E092494CBFEFC5A294C6CD85, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:22,243 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791781907_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:22,243 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:22,244 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791781907_0001_00_000002
2016-01-14 17:16:22,245 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791781907_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:22,247 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791781907_0001_00_000002
2016-01-14 17:16:22,248 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 17:16:22,249 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791781907_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:22,251 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:22,251 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791781907_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791781907_0001_00_000002, launchTime=1452791782250
2016-01-14 17:16:22,252 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791781907_0001_00_000002
2016-01-14 17:16:22,253 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791781907_0001_1_01_000000_0] to container: [container_1452791781907_0001_00_000002]
2016-01-14 17:16:22,253 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791781907_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:22,254 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791781907_0001_1_01_000000_0] started. Is using containerId: [container_1452791781907_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:22,254 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:22,254 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791781907_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B99E522F96164E14BBF172676F2D083D, taskAttemptId=attempt_1452791781907_0001_1_01_000000_0, startTime=1452791782254, containerId=container_1452791781907_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791781907_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:22,254 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791781907_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:22,255 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791781907_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:22,255 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791781907_0001_00_000002 given task: attempt_1452791781907_0001_1_01_000000_0
2016-01-14 17:16:22,255 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791781907_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791781907_0001_1_01_000000_0
2016-01-14 17:16:22 Starting to run new task attempt: attempt_1452791781907_0001_1_01_000000_0
2016-01-14 17:16:22,256 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791781907_0001_1_01_000000_0
2016-01-14 17:16:22,256 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:22,256 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:22,256 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:22,256 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:22,256 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:22,256 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:22,257 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [E9044769E91741B89DDA6C94FE9D4DFA/AA31AD92EBAC41A5A1398BCF6AD382E1] every/(1/1) ...lyplatform/-1_'field'_ALL, VertexName: B99E522F96164E14BBF172676F2D083D, VertexParallelism: 1, TaskAttemptID:attempt_1452791781907_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D42A4A5819E64448AC3DEE386CBA863E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=447C12C2E092494CBFEFC5A294C6CD85, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:22,257 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:16:22,257 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791781907_0001_1_01_000000_0
2016-01-14 17:16:22,257 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:22,257 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:22,258 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=D42A4A5819E64448AC3DEE386CBA863E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:22,258 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:22,259 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:22,259 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: D42A4A5819E64448AC3DEE386CBA863E
2016-01-14 17:16:22,261 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:22,262 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:22,262 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: D42A4A5819E64448AC3DEE386CBA863E
2016-01-14 17:16:22,263 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: D42A4A5819E64448AC3DEE386CBA863E
2016-01-14 17:16:22,262 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:22,263 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=447C12C2E092494CBFEFC5A294C6CD85, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:22,263 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791781907_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:22,265 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:22,266 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 447C12C2E092494CBFEFC5A294C6CD85
2016-01-14 17:16:22,266 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:22,266 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 447C12C2E092494CBFEFC5A294C6CD85
2016-01-14 17:16:22,280 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B99E522F96164E14BBF172676F2D083D, ordinal: 1
2016-01-14 17:16:22,280 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B99E522F96164E14BBF172676F2D083D, mem on start (mb), free: 123, total: 483, max: 672
2016-01-14 17:16:22,281 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:22,281 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:22,281 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:22,286 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:22,286 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 447C12C2E092494CBFEFC5A294C6CD85
2016-01-14 17:16:22,286 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:22,286 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:22,286 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:22,286 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:22,287 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:22,287 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:22,287 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:22,287 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, D42A4A5819E64448AC3DEE386CBA863E, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 17:16:22,288 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 447C12C2E092494CBFEFC5A294C6CD85, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:22,288 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:22,288 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: D42A4A5819E64448AC3DEE386CBA863E being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:22,288 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: D42A4A5819E64448AC3DEE386CBA863E
2016-01-14 17:16:22,290 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:22,290 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:22,291 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:22,291 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:22,291 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: D42A4A5819E64448AC3DEE386CBA863E with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:22,292 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 17:16:22,292 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:22,292 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:22,294 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: D42A4A5819E64448AC3DEE386CBA863E
2016-01-14 17:16:22,295 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:22,297 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791781907_0001_1_01_000000_0
2016-01-14 17:16:22,297 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791781907_0001_1_00_000000_0_10021, runDuration: 56179, ]
2016-01-14 17:16:22,299 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791781907_0001_1_00_000000_0_10021] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791782299, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 17:16:22,300 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : D42A4A5819E64448AC3DEE386CBA863E
2016-01-14 17:16:22,300 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:22,301 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:22,301 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [D42A4A5819E64448AC3DEE386CBA863E] #4 in 3ms
2016-01-14 17:16:22,302 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [D42A4A5819E64448AC3DEE386CBA863E] #4, status:false, isInterrupted:false
2016-01-14 17:16:22,302 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 183062C428334711978011C737EA018B
2016-01-14 17:16:22,303 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:22,304 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:22,304 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:22,304 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:22,304 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_ALL"], id: 447C12C2E092494CBFEFC5A294C6CD85
2016-01-14 17:16:22,305 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:22,305 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : D42A4A5819E64448AC3DEE386CBA863E
2016-01-14 17:16:22,305 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_ALL"]
2016-01-14 17:16:22,308 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:22,309 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 183062C428334711978011C737EA018B, for 1 inputs
2016-01-14 17:16:22,309 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B99E522F96164E14BBF172676F2D083D, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:22,310 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@73b732a; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@5005ce1b
2016-01-14 17:16:22,315 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:22,315 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:22,316 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791781907_0001_1_01_000000_0
2016-01-14 17:16:22,316 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791781907_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:22,318 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791781907_0001_1_01_000000_0
2016-01-14 17:16:22,318 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B99E522F96164E14BBF172676F2D083D, mem on close (mb), free: 120, total: 483, max: 672
2016-01-14 17:16:22,318 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: D42A4A5819E64448AC3DEE386CBA863E
2016-01-14 17:16:22,319 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:22,319 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:22,319 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:22,319 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:22,320 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:22 Completed running task attempt: attempt_1452791781907_0001_1_01_000000_0
2016-01-14 17:16:22,321 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791781907_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:22,325 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791781907_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B99E522F96164E14BBF172676F2D083D, taskAttemptId=attempt_1452791781907_0001_1_01_000000_0, startTime=1452791782254, finishTime=1452791782325, timeTaken=71, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=506986496, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791782309, Process_Duration=9, Process_End_Time=1452791782318, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:22,325 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:22,325 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791781907_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:22,326 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:22,326 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791781907_0001_1_01_000000_0
2016-01-14 17:16:22,327 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:22,327 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791781907_0001_1][Event:TASK_FINISHED]: vertexName=B99E522F96164E14BBF172676F2D083D, taskId=task_1452791781907_0001_1_01_000000, startTime=1452791782254, finishTime=1452791782326, timeTaken=72, status=SUCCEEDED, successfulAttemptID=attempt_1452791781907_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=506986496, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791782309, Process_Duration=9, Process_End_Time=1452791782318, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:22,327 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791781907_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:22,327 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791781907_0001_1_01 [B99E522F96164E14BBF172676F2D083D] : 1
2016-01-14 17:16:22,327 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791781907_0001_1_01 [B99E522F96164E14BBF172676F2D083D]
2016-01-14 17:16:22,329 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:22,329 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791781907_0001_1][Event:VERTEX_FINISHED]: vertexName=B99E522F96164E14BBF172676F2D083D, vertexId=vertex_1452791781907_0001_1_01, initRequestedTime=1452791782114, initedTime=1452791782128, startRequestedTime=1452791782141, startedTime=1452791782141, finishTime=1452791782328, timeTaken=187, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=506986496, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791782309, Process_Duration=9, Process_End_Time=1452791782318, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452791782254, firstTasksToStart=[ task_1452791781907_0001_1_01_000000 ], lastTaskFinishTime=1452791782325, lastTasksToFinish=[ task_1452791781907_0001_1_01_000000 ], minTaskDuration=71, maxTaskDuration=71, avgTaskDuration=71.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791781907_0001_1_01_000000 ], longestDurationTasks=[ task_1452791781907_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:22,329 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791781907_0001_1_01 [B99E522F96164E14BBF172676F2D083D] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:22,330 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=506986496, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791782309, Process_Duration=9, Process_End_Time=1452791782318, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 17:16:22,330 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791781907_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:22,331 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791781907_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791781907_0001_00_000002, stoppedTime=1452791782331, exitStatus=0
2016-01-14 17:16:22,331 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:22,331 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:22,332 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:22,331 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791781907_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:22,332 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791781907_0001_00_000002
2016-01-14 17:16:22,332 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791781907_0001_1_01 [B99E522F96164E14BBF172676F2D083D] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:22,332 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791781907_0001_1
2016-01-14 17:16:22,333 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791781907_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452791781907_0001_1
2016-01-14 17:16:22,333 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452791781907_0001_1_00 [D42A4A5819E64448AC3DEE386CBA863E]
2016-01-14 17:16:22,333 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 447C12C2E092494CBFEFC5A294C6CD85 for vertex: vertex_1452791781907_0001_1_01 [B99E522F96164E14BBF172676F2D083D]
2016-01-14 17:16:22,332 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791781907_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 17:16:22,336 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791781907_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:22,336 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791781907_0001_00_000002, Exiting
2016-01-14 17:16:22,336 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791781907_0001_00_000002
2016-01-14 17:16:22,336 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791781907_0001_00_000002
2016-01-14 17:16:22,337 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791781907_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:22,337 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791781907_0001_00_000002 completed successfully
2016-01-14 17:16:22,347 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791781907_0001_1, queueSize=0
2016-01-14 17:16:22,347 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:22,348 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791781907_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791781907_0001_1, startTime=1452791782111, finishTime=1452791782344, timeTaken=233, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=571, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1013972992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905583564521, Process_Duration=12, Process_End_Time=2905583564533, Read_Duration=4, Tuples_Read=40, Tuples_Written=40, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 17:16:22,349 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791781907_0001_1 finished with state: SUCCEEDED
2016-01-14 17:16:22,349 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791781907_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:22,350 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452791781907_0001_00_000002
2016-01-14 17:16:22,350 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791781907_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:16:22 Completed Dag: dag_1452791781907_0001_1
2016-01-14 17:16:22,352 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791781907_0001_1, dagState=SUCCEEDED
2016-01-14 17:16:22,352 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791781907_0001_1_post
2016-01-14 17:16:22,353 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:22,353 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:22,358 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...lyplatform/-1_'field'_ALL, applicationId=application_1452791781907_0001
2016-01-14 17:16:22,361 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:22,361 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:22,361 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:22,361 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:22,361 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:22,361 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:22,362 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:22,364 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:22,364 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:22,364 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:22,364 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:22,364 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:22,365 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:22,369 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E9044769E91741B89DDA6C94FE9D4DFA/AA31AD92EBAC41A5A1398BCF6AD382E1/.tez/application_1452791781907_0001
2016-01-14 17:16:22,370 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.489
2016-01-14 17:16:22,459 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_'field'_ALL
2016-01-14 17:16:22,488 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:22,489 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:22,529 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.008
2016-01-14 17:16:22,534 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.013
2016-01-14 17:16:22,534 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 17:16:22,535 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 17:16:22,535 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_'field'_REPLACE
2016-01-14 17:16:22,558 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:22,558 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:22,611 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.025
2016-01-14 17:16:22,620 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.034
2016-01-14 17:16:22,620 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:22,621 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:22,621 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:22,637 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:22,644 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_SWAP
2016-01-14 17:16:22,699 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:22,699 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:22,700 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_SWAP"]
2016-01-14 17:16:22,700 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:22,700 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:22,701 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:22,702 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...yplatform/-1_'field'_SWAP
2016-01-14 17:16:22,703 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:22,718 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:22,723 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:22,723 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:22,724 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:22,733 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/64CF3F95913E448E98938F9198D4480C/CCD28B2F6DE643C584941832A5F92C81/.tez/application_1452791782723_0001 doesn't exist and is created
2016-01-14 17:16:22,768 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:22,768 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:22,769 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/64CF3F95913E448E98938F9198D4480C/CCD28B2F6DE643C584941832A5F92C81/.tez/application_1452791782723_0001
2016-01-14 17:16:22,773 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791782723_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:22,786 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:22,787 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:22,789 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:22,789 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:22,790 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/64CF3F95913E448E98938F9198D4480C/CCD28B2F6DE643C584941832A5F92C81/.tez/application_1452791782723_0001/localmode-log-dir
2016-01-14 17:16:22,790 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/64CF3F95913E448E98938F9198D4480C/CCD28B2F6DE643C584941832A5F92C81/.tez/application_1452791782723_0001/localmode-log-dir/history.txt.appattempt_1452791782723_0001_000000, maxErrors=10
2016-01-14 17:16:22,790 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:22,791 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791782723_0001_000000, appSubmitTime=1452791782773, launchTime=1452791782773
2016-01-14 17:16:22,795 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:22,797 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:22,801 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:22,801 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:22,801 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:38927
2016-01-14 17:16:22,803 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:22,805 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791782723_0001_000000, startTime=1452791782805
2016-01-14 17:16:22,805 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:22,805 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:22,869 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:22,869 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:22,869 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...yplatform/-1_'field'_SWAP, applicationId=application_1452791782723_0001, dagName=[64CF3F95913E448E98938F9198D4480C/CCD28B2F6DE643C584941832A5F92C81] every/(1/1) ...yplatform/-1_'field'_SWAP
2016-01-14 17:16:22,887 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [64CF3F95913E448E98938F9198D4480C/CCD28B2F6DE643C584941832A5F92C81] every/(1/1) ...yplatform/-1_'field'_SWAP
2016-01-14 17:16:22,888 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791782723_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/64CF3F95913E448E98938F9198D4480C/CCD28B2F6DE643C584941832A5F92C81/.tez/application_1452791782723_0001/localmode-log-dir/dag_1452791782723_0001_1.dot
2016-01-14 17:16:22,890 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/64CF3F95913E448E98938F9198D4480C/CCD28B2F6DE643C584941832A5F92C81/.tez/application_1452791782723_0001/localmode-log-dir/dag_1452791782723_0001_1-tez-dag.pb.txt
2016-01-14 17:16:22,918 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791782723_0001_1
2016-01-14 17:16:22,918 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:22,918 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [64CF3F95913E448E98938F9198D4480C/CCD28B2F6DE643C584941832A5F92C81] every/(1/1) ...yplatform/-1_'field'_SWAP
2016-01-14 17:16:22 Running Dag: dag_1452791782723_0001_1
2016-01-14 17:16:22,939 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791782723_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791782723_0001_1, submitTime=1452791782887
2016-01-14 17:16:22,939 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:22,941 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex DBB05B87223D43F7A1A8ACE8F17BCC3E
2016-01-14 17:16:22,941 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:22,942 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:22,943 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791782723_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791782723_0001_1, initTime=1452791782940
2016-01-14 17:16:22,943 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791782723_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:22,944 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791782723_0001_1][Event:DAG_STARTED]: dagID=dag_1452791782723_0001_1, startTime=1452791782944
2016-01-14 17:16:22,944 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:22,944 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:22,945 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791782723_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:22,945 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: E4782397C430464ABB16C16F79EA95E1 : {84E8A6CFAB134CED96CB99220CBE74DE={InputName=84E8A6CFAB134CED96CB99220CBE74DE}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:22,945 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 84E8A6CFAB134CED96CB99220CBE74DE, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:22,945 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791782723_0001_1_00 [E4782397C430464ABB16C16F79EA95E1]
2016-01-14 17:16:22,945 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791782723_0001_1_00 [E4782397C430464ABB16C16F79EA95E1]
2016-01-14 17:16:22,946 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791782723_0001_1_00 [E4782397C430464ABB16C16F79EA95E1]
2016-01-14 17:16:22,946 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791782723_0001_1_00 [E4782397C430464ABB16C16F79EA95E1]. Starting root input initializers: 1
2016-01-14 17:16:22,946 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791782723_0001_1_00 [E4782397C430464ABB16C16F79EA95E1] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:22,947 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...yplatform/-1_'field'_SWAP, applicationId=application_1452791782723_0001, dagName=[64CF3F95913E448E98938F9198D4480C/CCD28B2F6DE643C584941832A5F92C81] every/(1/1) ...yplatform/-1_'field'_SWAP
2016-01-14 17:16:22,947 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791782723_0001_1_01 [DBB05B87223D43F7A1A8ACE8F17BCC3E]
2016-01-14 17:16:22,947 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 84E8A6CFAB134CED96CB99220CBE74DE on vertex vertex_1452791782723_0001_1_00 [E4782397C430464ABB16C16F79EA95E1]
2016-01-14 17:16:22,949 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:22,949 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 84E8A6CFAB134CED96CB99220CBE74DE asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:22,950 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:22,951 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:22,951 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791782723_0001_1_01 [DBB05B87223D43F7A1A8ACE8F17BCC3E]
2016-01-14 17:16:22,952 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:22,952 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791782723_0001_1_01 [DBB05B87223D43F7A1A8ACE8F17BCC3E]
2016-01-14 17:16:22,952 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791782723_0001_1_01 [DBB05B87223D43F7A1A8ACE8F17BCC3E]
2016-01-14 17:16:22,953 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=6A1032AADE86420D92AE9799A5E844B3, vertexId=vertex_1452791782723_0001_1_01 [DBB05B87223D43F7A1A8ACE8F17BCC3E], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:22,953 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=6A1032AADE86420D92AE9799A5E844B3, vertexId=vertex_1452791782723_0001_1_01 [DBB05B87223D43F7A1A8ACE8F17BCC3E]
2016-01-14 17:16:22,955 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=6A1032AADE86420D92AE9799A5E844B3, vertexName=DBB05B87223D43F7A1A8ACE8F17BCC3E, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:22,956 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=6A1032AADE86420D92AE9799A5E844B3, vertexId=vertex_1452791782723_0001_1_01 [DBB05B87223D43F7A1A8ACE8F17BCC3E]
2016-01-14 17:16:22,957 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:22,959 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:22,960 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 84E8A6CFAB134CED96CB99220CBE74DE on vertex vertex_1452791782723_0001_1_00 [E4782397C430464ABB16C16F79EA95E1]
2016-01-14 17:16:22,960 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791782723_0001_1][Event:VERTEX_INITIALIZED]: vertexName=DBB05B87223D43F7A1A8ACE8F17BCC3E, vertexId=vertex_1452791782723_0001_1_01, initRequestedTime=1452791782947, initedTime=1452791782959, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:22,961 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791782723_0001_1_01 [DBB05B87223D43F7A1A8ACE8F17BCC3E] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:22,961 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791782723_0001_1_00 [E4782397C430464ABB16C16F79EA95E1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:22,961 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {84E8A6CFAB134CED96CB99220CBE74DE=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:22,962 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791782723_0001_1_00 [E4782397C430464ABB16C16F79EA95E1] parallelism set to 1
2016-01-14 17:16:22,962 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791782723_0001_1_00 [E4782397C430464ABB16C16F79EA95E1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:22,962 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791782723_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791782723_0001_1_00, eventCount=1
2016-01-14 17:16:22,963 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:22,964 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791782723_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E4782397C430464ABB16C16F79EA95E1, vertexId=vertex_1452791782723_0001_1_00, initRequestedTime=1452791782945, initedTime=1452791782963, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:22,964 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791782723_0001_1_00 [E4782397C430464ABB16C16F79EA95E1] with distanceFromRoot: 0
2016-01-14 17:16:22,964 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791782723_0001_1_00 [E4782397C430464ABB16C16F79EA95E1] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:22,964 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in E4782397C430464ABB16C16F79EA95E1
2016-01-14 17:16:22,964 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791782723_0001_1_00 [E4782397C430464ABB16C16F79EA95E1]
2016-01-14 17:16:22,965 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:22,965 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791782723_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791782723_0001_1_00, startRequestedTime=1452791782964, startedTime=1452791782964
2016-01-14 17:16:22,966 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791782723_0001_1_00 [E4782397C430464ABB16C16F79EA95E1] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:22,966 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:22,967 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791782723_0001_1][Event:TASK_STARTED]: vertexName=E4782397C430464ABB16C16F79EA95E1, taskId=task_1452791782723_0001_1_00_000000, scheduledTime=1452791782966, launchTime=1452791782966
2016-01-14 17:16:22,967 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791782723_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:22,967 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791782723_0001_1_00 for vertex: vertex_1452791782723_0001_1_01 [DBB05B87223D43F7A1A8ACE8F17BCC3E] numStartedSources: 1 numSources: 1
2016-01-14 17:16:22,967 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791782723_0001_1_01 [DBB05B87223D43F7A1A8ACE8F17BCC3E] with distanceFromRoot: 1
2016-01-14 17:16:22,968 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791782723_0001_1_00 [E4782397C430464ABB16C16F79EA95E1]
2016-01-14 17:16:22,968 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: E4782397C430464ABB16C16F79EA95E1 in vertex: DBB05B87223D43F7A1A8ACE8F17BCC3E
2016-01-14 17:16:22,968 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: DBB05B87223D43F7A1A8ACE8F17BCC3E with 1 source tasks and 1 pending tasks
2016-01-14 17:16:22,969 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:22,969 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791782723_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791782723_0001_1_01, startRequestedTime=1452791782968, startedTime=1452791782968
2016-01-14 17:16:22,969 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791782723_0001_1_01 [DBB05B87223D43F7A1A8ACE8F17BCC3E] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:22,970 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [64CF3F95913E448E98938F9198D4480C/CCD28B2F6DE643C584941832A5F92C81] every/(1/1) ...yplatform/-1_'field'_SWAP, VertexName: E4782397C430464ABB16C16F79EA95E1, VertexParallelism: 1, TaskAttemptID:attempt_1452791782723_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=84E8A6CFAB134CED96CB99220CBE74DE, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=DBB05B87223D43F7A1A8ACE8F17BCC3E, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:22,970 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791782723_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:22,970 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:22,971 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:22,980 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:22,980 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:22,980 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791782723_0001_00_000001
2016-01-14 17:16:22,981 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791782723_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:22,982 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791782723_0001, with dag id: dag_1452791782723_0001_1
2016-01-14 17:16:22,983 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791782723_0001_00_000001
2016-01-14 17:16:22,984 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:22,985 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791782723_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:22,985 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:22,986 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791782723_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791782723_0001_00_000001, launchTime=1452791782985
2016-01-14 17:16:22,987 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791782723_0001_00_000001
2016-01-14 17:16:22,991 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791782723_0001_1_00_000000_0] to container: [container_1452791782723_0001_00_000001]
2016-01-14 17:16:22,992 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791782723_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:22,993 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791782723_0001_1_00_000000_0] started. Is using containerId: [container_1452791782723_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:22,993 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791782723_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E4782397C430464ABB16C16F79EA95E1, taskAttemptId=attempt_1452791782723_0001_1_00_000000_0, startTime=1452791782992, containerId=container_1452791782723_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791782723_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:22,993 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791782723_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:22,993 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791782723_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:22,994 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:22,994 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791782723_0001_00_000001 given task: attempt_1452791782723_0001_1_00_000000_0
2016-01-14 17:16:22,995 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791782723_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791782723_0001_1_00_000000_0
2016-01-14 17:16:22 Starting to run new task attempt: attempt_1452791782723_0001_1_00_000000_0
2016-01-14 17:16:22,995 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791782723_0001_1_00_000000_0
2016-01-14 17:16:22,995 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:22,996 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:22,996 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:22,996 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:22,996 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:22,996 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:22,996 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [64CF3F95913E448E98938F9198D4480C/CCD28B2F6DE643C584941832A5F92C81] every/(1/1) ...yplatform/-1_'field'_SWAP, VertexName: E4782397C430464ABB16C16F79EA95E1, VertexParallelism: 1, TaskAttemptID:attempt_1452791782723_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=84E8A6CFAB134CED96CB99220CBE74DE, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=DBB05B87223D43F7A1A8ACE8F17BCC3E, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:22,996 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:16:22,998 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791782723_0001_1_00_000000_0
2016-01-14 17:16:22,999 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:23,000 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:23,000 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791782723_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:23,002 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=84E8A6CFAB134CED96CB99220CBE74DE, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:23,002 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:23,002 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 84E8A6CFAB134CED96CB99220CBE74DE
2016-01-14 17:16:23,002 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 84E8A6CFAB134CED96CB99220CBE74DE
2016-01-14 17:16:23,002 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=DBB05B87223D43F7A1A8ACE8F17BCC3E, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:23,002 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:23,006 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:23,006 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: DBB05B87223D43F7A1A8ACE8F17BCC3E
2016-01-14 17:16:23,006 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:23,007 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 84E8A6CFAB134CED96CB99220CBE74DE
2016-01-14 17:16:23,007 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 84E8A6CFAB134CED96CB99220CBE74DE
2016-01-14 17:16:23,007 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:23,012 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:23,012 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: DBB05B87223D43F7A1A8ACE8F17BCC3E
2016-01-14 17:16:23,013 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: DBB05B87223D43F7A1A8ACE8F17BCC3E
2016-01-14 17:16:23,018 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E4782397C430464ABB16C16F79EA95E1, ordinal: 0
2016-01-14 17:16:23,018 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E4782397C430464ABB16C16F79EA95E1, mem on start (mb), free: 231, total: 490, max: 672
2016-01-14 17:16:23,019 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:23,019 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:23,019 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:23,019 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:23,019 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:23,019 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:23,020 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:23,020 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:23,020 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:23,021 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 17:16:23,022 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 84E8A6CFAB134CED96CB99220CBE74DE, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:23,022 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, DBB05B87223D43F7A1A8ACE8F17BCC3E, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:23,023 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:23,023 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 84E8A6CFAB134CED96CB99220CBE74DE
2016-01-14 17:16:23,023 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:23,023 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 84E8A6CFAB134CED96CB99220CBE74DE
2016-01-14 17:16:23,023 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 84E8A6CFAB134CED96CB99220CBE74DE being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:23,024 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:23,024 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:23,024 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:23,025 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791782723_0001_1_00_000000_0
2016-01-14 17:16:23,029 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:23,029 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 84E8A6CFAB134CED96CB99220CBE74DE
2016-01-14 17:16:23,029 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: E942C51875C1421CBB94E9EEB3084E92
2016-01-14 17:16:23,030 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] E942C51875C1421CBB94E9EEB3084E92
2016-01-14 17:16:23,030 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@43495ba4; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@51f8e7f; comparator=cascading.tuple.hadoop.util.TupleComparator@5ac43679; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:23,030 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:23,031 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:23,031 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:23,032 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:23,032 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@c25228f
2016-01-14 17:16:23,035 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@c25228f
2016-01-14 17:16:23,035 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:23,035 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:23,045 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:23,045 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:23,046 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:23,046 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:23,046 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:23,046 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:23,047 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:23,047 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E4782397C430464ABB16C16F79EA95E1, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:23,050 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791782723_0001_1_00_000000_0
2016-01-14 17:16:23,050 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E4782397C430464ABB16C16F79EA95E1, mem on close (mb), free: 129, total: 490, max: 672
2016-01-14 17:16:23,050 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:23,051 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:23,051 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:23,051 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:23,055 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:23 Completed running task attempt: attempt_1452791782723_0001_1_00_000000_0
2016-01-14 17:16:23,067 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791782723_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:23,069 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791782723_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791782723_0001_1_00, eventCount=1
2016-01-14 17:16:23,069 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791783047, Process_Duration=3, Process_End_Time=1452791783050, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:23,070 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:23,070 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:23,070 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791782723_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E4782397C430464ABB16C16F79EA95E1, taskAttemptId=attempt_1452791782723_0001_1_00_000000_0, startTime=1452791782992, finishTime=1452791783069, timeTaken=77, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791783047, Process_Duration=3, Process_End_Time=1452791783050, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:23,070 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:23,070 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791782723_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:23,070 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791782723_0001_00_000001
2016-01-14 17:16:23,071 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:23,071 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791782723_0001_1_00_000000_0
2016-01-14 17:16:23,075 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:23,075 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791782723_0001_1][Event:TASK_FINISHED]: vertexName=E4782397C430464ABB16C16F79EA95E1, taskId=task_1452791782723_0001_1_00_000000, startTime=1452791782992, finishTime=1452791783074, timeTaken=82, status=SUCCEEDED, successfulAttemptID=attempt_1452791782723_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791783047, Process_Duration=3, Process_End_Time=1452791783050, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:23,075 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791782723_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:23,075 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791782723_0001_1_00 [E4782397C430464ABB16C16F79EA95E1] : 1
2016-01-14 17:16:23,075 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791782723_0001_1_00 [E4782397C430464ABB16C16F79EA95E1]
2016-01-14 17:16:23,077 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:23,077 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791782723_0001_1][Event:VERTEX_FINISHED]: vertexName=E4782397C430464ABB16C16F79EA95E1, vertexId=vertex_1452791782723_0001_1_00, initRequestedTime=1452791782945, initedTime=1452791782963, startRequestedTime=1452791782964, startedTime=1452791782964, finishTime=1452791783075, timeTaken=111, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791783047, Process_Duration=3, Process_End_Time=1452791783050, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791782992, firstTasksToStart=[ task_1452791782723_0001_1_00_000000 ], lastTaskFinishTime=1452791783069, lastTasksToFinish=[ task_1452791782723_0001_1_00_000000 ], minTaskDuration=77, maxTaskDuration=77, avgTaskDuration=77.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791782723_0001_1_00_000000 ], longestDurationTasks=[ task_1452791782723_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:23,077 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791782723_0001_1_00 [E4782397C430464ABB16C16F79EA95E1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:23,077 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791782723_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:23,078 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791782723_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791782723_0001_00_000001, stoppedTime=1452791783078, exitStatus=0
2016-01-14 17:16:23,078 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:23,078 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791782723_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:23,078 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791782723_0001_1_01 [DBB05B87223D43F7A1A8ACE8F17BCC3E] attempt: attempt_1452791782723_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:23,078 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: DBB05B87223D43F7A1A8ACE8F17BCC3E
2016-01-14 17:16:23,078 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791782723_0001_1_01 [DBB05B87223D43F7A1A8ACE8F17BCC3E]
2016-01-14 17:16:23,079 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791782723_0001_1_00 [E4782397C430464ABB16C16F79EA95E1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:23,079 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791782723_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:23,079 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791782723_0001_1][Event:TASK_STARTED]: vertexName=DBB05B87223D43F7A1A8ACE8F17BCC3E, taskId=task_1452791782723_0001_1_01_000000, scheduledTime=1452791783079, launchTime=1452791783079
2016-01-14 17:16:23,080 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791782723_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:23,080 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791782723_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:23,080 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791782723_0001_1_01 [DBB05B87223D43F7A1A8ACE8F17BCC3E]
2016-01-14 17:16:23,080 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [64CF3F95913E448E98938F9198D4480C/CCD28B2F6DE643C584941832A5F92C81] every/(1/1) ...yplatform/-1_'field'_SWAP, VertexName: DBB05B87223D43F7A1A8ACE8F17BCC3E, VertexParallelism: 1, TaskAttemptID:attempt_1452791782723_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E4782397C430464ABB16C16F79EA95E1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=6A1032AADE86420D92AE9799A5E844B3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:23,080 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791782723_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:23,081 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:23,081 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:23,082 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791782723_0001_00_000002
2016-01-14 17:16:23,082 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791782723_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:23,084 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791782723_0001_00_000002
2016-01-14 17:16:23,084 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:23,084 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791782723_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:23,085 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791782723_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:23,085 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791782723_0001_00_000001, Exiting
2016-01-14 17:16:23,085 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791782723_0001_00_000001
2016-01-14 17:16:23,085 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791782723_0001_00_000001
2016-01-14 17:16:23,090 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791782723_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:23,090 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791782723_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791782723_0001_00_000002, launchTime=1452791783090
2016-01-14 17:16:23,091 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791782723_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:23,092 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791782723_0001_00_000001 completed successfully
2016-01-14 17:16:23,092 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791782723_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:23,090 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:23,091 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791782723_0001_00_000002
2016-01-14 17:16:23,094 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791782723_0001_1_01_000000_0] to container: [container_1452791782723_0001_00_000002]
2016-01-14 17:16:23,094 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791782723_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:23,094 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791782723_0001_00_000002 given task: attempt_1452791782723_0001_1_01_000000_0
2016-01-14 17:16:23,094 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791782723_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791782723_0001_1_01_000000_0
2016-01-14 17:16:23,095 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791782723_0001_1_01_000000_0] started. Is using containerId: [container_1452791782723_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:23 Starting to run new task attempt: attempt_1452791782723_0001_1_01_000000_0
2016-01-14 17:16:23,096 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791782723_0001_1_01_000000_0
2016-01-14 17:16:23,096 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:23,096 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:23,096 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:23,096 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:23,097 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:23,097 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:23,097 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [64CF3F95913E448E98938F9198D4480C/CCD28B2F6DE643C584941832A5F92C81] every/(1/1) ...yplatform/-1_'field'_SWAP, VertexName: DBB05B87223D43F7A1A8ACE8F17BCC3E, VertexParallelism: 1, TaskAttemptID:attempt_1452791782723_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E4782397C430464ABB16C16F79EA95E1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=6A1032AADE86420D92AE9799A5E844B3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:23,096 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:23,097 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:16:23,097 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791782723_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=DBB05B87223D43F7A1A8ACE8F17BCC3E, taskAttemptId=attempt_1452791782723_0001_1_01_000000_0, startTime=1452791783095, containerId=container_1452791782723_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791782723_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:23,099 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791782723_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:23,099 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791782723_0001_1_01_000000_0
2016-01-14 17:16:23,104 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:23,104 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:23,104 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791782723_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:23,105 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791782723_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:23,105 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=E4782397C430464ABB16C16F79EA95E1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:23,106 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=6A1032AADE86420D92AE9799A5E844B3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:23,106 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:23,106 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:23,107 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:23,107 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 6A1032AADE86420D92AE9799A5E844B3
2016-01-14 17:16:23,108 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:23,108 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 6A1032AADE86420D92AE9799A5E844B3
2016-01-14 17:16:23,110 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:23,113 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: E4782397C430464ABB16C16F79EA95E1
2016-01-14 17:16:23,115 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:23,116 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:23,117 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: E4782397C430464ABB16C16F79EA95E1
2016-01-14 17:16:23,118 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: E4782397C430464ABB16C16F79EA95E1
2016-01-14 17:16:23,120 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: DBB05B87223D43F7A1A8ACE8F17BCC3E, ordinal: 1
2016-01-14 17:16:23,120 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DBB05B87223D43F7A1A8ACE8F17BCC3E, mem on start (mb), free: 123, total: 490, max: 672
2016-01-14 17:16:23,120 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:23,120 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:23,120 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:23,124 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:23,124 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 6A1032AADE86420D92AE9799A5E844B3
2016-01-14 17:16:23,124 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:23,124 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:23,125 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:23,125 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:23,125 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:23,125 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:23,125 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:23,125 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 6A1032AADE86420D92AE9799A5E844B3, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:23,126 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, E4782397C430464ABB16C16F79EA95E1, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 17:16:23,126 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:23,126 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: E4782397C430464ABB16C16F79EA95E1 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:23,126 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: E4782397C430464ABB16C16F79EA95E1
2016-01-14 17:16:23,126 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:23,127 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:23,128 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:23,128 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:23,128 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: E4782397C430464ABB16C16F79EA95E1 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:23,129 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 17:16:23,129 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:23,129 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:23,130 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: E4782397C430464ABB16C16F79EA95E1
2016-01-14 17:16:23,131 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:23,131 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791782723_0001_1_01_000000_0
2016-01-14 17:16:23,134 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791782723_0001_1_00_000000_0_10027, runDuration: 50967, ]
2016-01-14 17:16:23,135 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:23,136 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791782723_0001_1_00_000000_0_10027] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791783136, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 17:16:23,138 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : E4782397C430464ABB16C16F79EA95E1
2016-01-14 17:16:23,138 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: E942C51875C1421CBB94E9EEB3084E92
2016-01-14 17:16:23,138 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_SWAP"], id: 6A1032AADE86420D92AE9799A5E844B3
2016-01-14 17:16:23,139 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_SWAP"]
2016-01-14 17:16:23,139 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] E942C51875C1421CBB94E9EEB3084E92, for 1 inputs
2016-01-14 17:16:23,138 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:23,140 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [E4782397C430464ABB16C16F79EA95E1] #5 in 5ms
2016-01-14 17:16:23,140 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [E4782397C430464ABB16C16F79EA95E1] #5, status:false, isInterrupted:false
2016-01-14 17:16:23,141 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:23,141 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:23,141 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:23,142 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:23,142 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:23,142 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : E4782397C430464ABB16C16F79EA95E1
2016-01-14 17:16:23,143 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:23,144 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: DBB05B87223D43F7A1A8ACE8F17BCC3E, all 1 inputs ready in: 00:00:00.003
2016-01-14 17:16:23,144 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@3d98c5ad; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@78da0072
2016-01-14 17:16:23,150 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:23,151 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:23,152 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791782723_0001_1_01_000000_0
2016-01-14 17:16:23,153 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791782723_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:23,154 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791782723_0001_1_01_000000_0
2016-01-14 17:16:23,154 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DBB05B87223D43F7A1A8ACE8F17BCC3E, mem on close (mb), free: 119, total: 490, max: 672
2016-01-14 17:16:23,155 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: E4782397C430464ABB16C16F79EA95E1
2016-01-14 17:16:23,155 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:23,156 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:23,156 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:23,157 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:23,157 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:23 Completed running task attempt: attempt_1452791782723_0001_1_01_000000_0
2016-01-14 17:16:23,159 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791782723_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:23,162 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:23,162 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791782723_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=DBB05B87223D43F7A1A8ACE8F17BCC3E, taskAttemptId=attempt_1452791782723_0001_1_01_000000_0, startTime=1452791783095, finishTime=1452791783161, timeTaken=66, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=707, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791783144, Process_Duration=10, Process_End_Time=1452791783154, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:23,162 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791782723_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:23,162 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791782723_0001_1_01_000000_0
2016-01-14 17:16:23,163 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:23,163 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791782723_0001_1][Event:TASK_FINISHED]: vertexName=DBB05B87223D43F7A1A8ACE8F17BCC3E, taskId=task_1452791782723_0001_1_01_000000, startTime=1452791783095, finishTime=1452791783162, timeTaken=67, status=SUCCEEDED, successfulAttemptID=attempt_1452791782723_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=707, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791783144, Process_Duration=10, Process_End_Time=1452791783154, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:23,164 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791782723_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:23,164 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791782723_0001_1_01 [DBB05B87223D43F7A1A8ACE8F17BCC3E] : 1
2016-01-14 17:16:23,164 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791782723_0001_1_01 [DBB05B87223D43F7A1A8ACE8F17BCC3E]
2016-01-14 17:16:23,163 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:23,165 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:23,165 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791782723_0001_1][Event:VERTEX_FINISHED]: vertexName=DBB05B87223D43F7A1A8ACE8F17BCC3E, vertexId=vertex_1452791782723_0001_1_01, initRequestedTime=1452791782947, initedTime=1452791782959, startRequestedTime=1452791782968, startedTime=1452791782968, finishTime=1452791783164, timeTaken=196, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=707, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791783144, Process_Duration=10, Process_End_Time=1452791783154, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452791783095, firstTasksToStart=[ task_1452791782723_0001_1_01_000000 ], lastTaskFinishTime=1452791783161, lastTasksToFinish=[ task_1452791782723_0001_1_01_000000 ], minTaskDuration=66, maxTaskDuration=66, avgTaskDuration=66.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791782723_0001_1_01_000000 ], longestDurationTasks=[ task_1452791782723_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:23,166 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791782723_0001_1_01 [DBB05B87223D43F7A1A8ACE8F17BCC3E] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:23,166 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=707, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791783144, Process_Duration=10, Process_End_Time=1452791783154, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 17:16:23,167 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791782723_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:23,167 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:23,168 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:23,169 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791782723_0001_00_000002
2016-01-14 17:16:23,169 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791782723_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 17:16:23,170 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791782723_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:23,170 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791782723_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791782723_0001_00_000002, stoppedTime=1452791783169, exitStatus=0
2016-01-14 17:16:23,170 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:23,170 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791782723_0001_00_000002, Exiting
2016-01-14 17:16:23,171 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791782723_0001_00_000002
2016-01-14 17:16:23,171 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791782723_0001_00_000002
2016-01-14 17:16:23,173 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791782723_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:23,173 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791782723_0001_00_000002 completed successfully
2016-01-14 17:16:23,173 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791782723_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:23,174 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791782723_0001_1_01 [DBB05B87223D43F7A1A8ACE8F17BCC3E] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:23,174 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791782723_0001_1
2016-01-14 17:16:23,174 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791782723_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452791782723_0001_1
2016-01-14 17:16:23,175 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 6A1032AADE86420D92AE9799A5E844B3 for vertex: vertex_1452791782723_0001_1_01 [DBB05B87223D43F7A1A8ACE8F17BCC3E]
2016-01-14 17:16:23,187 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452791782723_0001_1_00 [E4782397C430464ABB16C16F79EA95E1]
2016-01-14 17:16:23,190 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791782723_0001_1, queueSize=0
2016-01-14 17:16:23,191 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:23,192 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791782723_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791782723_0001_1, startTime=1452791782944, finishTime=1452791783187, timeTaken=243, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=1040, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1028653056, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905583566191, Process_Duration=13, Process_End_Time=2905583566204, Read_Duration=2, Tuples_Read=40, Tuples_Written=40, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 17:16:23,192 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791782723_0001_1 finished with state: SUCCEEDED
2016-01-14 17:16:23,193 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791782723_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:23,193 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791782723_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:16:23,193 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452791782723_0001_00_000002
2016-01-14 17:16:23 Completed Dag: dag_1452791782723_0001_1
2016-01-14 17:16:23,194 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791782723_0001_1, dagState=SUCCEEDED
2016-01-14 17:16:23,194 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791782723_0001_1_post
2016-01-14 17:16:23,195 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:23,195 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:23,202 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...yplatform/-1_'field'_SWAP, applicationId=application_1452791782723_0001
2016-01-14 17:16:23,205 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:23,205 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:23,205 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:23,205 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:23,205 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:23,206 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:23,206 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:23,207 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:23,207 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:23,207 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:23,208 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:23,208 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:23,209 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:23,214 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/64CF3F95913E448E98938F9198D4480C/CCD28B2F6DE643C584941832A5F92C81/.tez/application_1452791782723_0001
2016-01-14 17:16:23,214 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.515
2016-01-14 17:16:23,302 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_'field'_SWAP
2016-01-14 17:16:23,328 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:23,328 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:23,392 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.021
2016-01-14 17:16:23,412 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.041
2016-01-14 17:16:23,413 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:23,413 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:23,414 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:23,432 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:23,440 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_-1
2016-01-14 17:16:23,493 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:23,494 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:23,494 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_-1"]
2016-01-14 17:16:23,494 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:23,494 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:23,494 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:23,495 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...blyplatform/-1_UNKNOWN_-1
2016-01-14 17:16:23,496 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:23,515 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:23,520 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:23,520 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:23,520 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:23,531 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/63CD34C4A54D416CA144CEBF2B5B0049/DF79026A731B48D3979A867A7AD3E43B/.tez/application_1452791783520_0001 doesn't exist and is created
2016-01-14 17:16:23,549 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:23,550 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:23,550 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/63CD34C4A54D416CA144CEBF2B5B0049/DF79026A731B48D3979A867A7AD3E43B/.tez/application_1452791783520_0001
2016-01-14 17:16:23,554 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791783520_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:23,566 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:23,568 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:23,570 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:23,571 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:23,571 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/63CD34C4A54D416CA144CEBF2B5B0049/DF79026A731B48D3979A867A7AD3E43B/.tez/application_1452791783520_0001/localmode-log-dir
2016-01-14 17:16:23,572 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/63CD34C4A54D416CA144CEBF2B5B0049/DF79026A731B48D3979A867A7AD3E43B/.tez/application_1452791783520_0001/localmode-log-dir/history.txt.appattempt_1452791783520_0001_000000, maxErrors=10
2016-01-14 17:16:23,573 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:23,573 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791783520_0001_000000, appSubmitTime=1452791783553, launchTime=1452791783554
2016-01-14 17:16:23,578 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:23,582 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:23,582 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:42809
2016-01-14 17:16:23,597 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:23,598 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:23,599 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:23,602 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791783520_0001_000000, startTime=1452791783602
2016-01-14 17:16:23,602 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:23,602 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:23,650 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:23,650 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:23,650 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...blyplatform/-1_UNKNOWN_-1, applicationId=application_1452791783520_0001, dagName=[63CD34C4A54D416CA144CEBF2B5B0049/DF79026A731B48D3979A867A7AD3E43B] every/(1/1) ...blyplatform/-1_UNKNOWN_-1
2016-01-14 17:16:23,671 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [63CD34C4A54D416CA144CEBF2B5B0049/DF79026A731B48D3979A867A7AD3E43B] every/(1/1) ...blyplatform/-1_UNKNOWN_-1
2016-01-14 17:16:23,672 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791783520_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/63CD34C4A54D416CA144CEBF2B5B0049/DF79026A731B48D3979A867A7AD3E43B/.tez/application_1452791783520_0001/localmode-log-dir/dag_1452791783520_0001_1.dot
2016-01-14 17:16:23,673 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/63CD34C4A54D416CA144CEBF2B5B0049/DF79026A731B48D3979A867A7AD3E43B/.tez/application_1452791783520_0001/localmode-log-dir/dag_1452791783520_0001_1-tez-dag.pb.txt
2016-01-14 17:16:23,690 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791783520_0001_1
2016-01-14 17:16:23,690 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:23,690 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [63CD34C4A54D416CA144CEBF2B5B0049/DF79026A731B48D3979A867A7AD3E43B] every/(1/1) ...blyplatform/-1_UNKNOWN_-1
2016-01-14 17:16:23 Running Dag: dag_1452791783520_0001_1
2016-01-14 17:16:23,708 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791783520_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791783520_0001_1, submitTime=1452791783671
2016-01-14 17:16:23,708 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:23,710 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex DD68E35AF41A4CA6AACCF6599A70F5E2
2016-01-14 17:16:23,710 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:23,711 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791783520_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791783520_0001_1, initTime=1452791783709
2016-01-14 17:16:23,712 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791783520_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:23,711 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:23,712 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791783520_0001_1][Event:DAG_STARTED]: dagID=dag_1452791783520_0001_1, startTime=1452791783712
2016-01-14 17:16:23,712 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:23,712 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:23,713 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791783520_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:23,713 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D5BAC4FF61A8486F8898DD3401238B1C : {29904394FEEA402EB30715E427A84ABF={InputName=29904394FEEA402EB30715E427A84ABF}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:23,714 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 29904394FEEA402EB30715E427A84ABF, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:23,714 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791783520_0001_1_00 [D5BAC4FF61A8486F8898DD3401238B1C]
2016-01-14 17:16:23,714 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791783520_0001_1_00 [D5BAC4FF61A8486F8898DD3401238B1C]
2016-01-14 17:16:23,714 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791783520_0001_1_00 [D5BAC4FF61A8486F8898DD3401238B1C]
2016-01-14 17:16:23,714 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791783520_0001_1_00 [D5BAC4FF61A8486F8898DD3401238B1C]. Starting root input initializers: 1
2016-01-14 17:16:23,715 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791783520_0001_1_00 [D5BAC4FF61A8486F8898DD3401238B1C] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:23,715 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791783520_0001_1_01 [DD68E35AF41A4CA6AACCF6599A70F5E2]
2016-01-14 17:16:23,716 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...blyplatform/-1_UNKNOWN_-1, applicationId=application_1452791783520_0001, dagName=[63CD34C4A54D416CA144CEBF2B5B0049/DF79026A731B48D3979A867A7AD3E43B] every/(1/1) ...blyplatform/-1_UNKNOWN_-1
2016-01-14 17:16:23,719 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:23,719 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791783520_0001_1_01 [DD68E35AF41A4CA6AACCF6599A70F5E2]
2016-01-14 17:16:23,719 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791783520_0001_1_01 [DD68E35AF41A4CA6AACCF6599A70F5E2]
2016-01-14 17:16:23,719 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791783520_0001_1_01 [DD68E35AF41A4CA6AACCF6599A70F5E2]
2016-01-14 17:16:23,719 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=2F2A1170585046319A32B83434535562, vertexId=vertex_1452791783520_0001_1_01 [DD68E35AF41A4CA6AACCF6599A70F5E2], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:23,719 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=2F2A1170585046319A32B83434535562, vertexId=vertex_1452791783520_0001_1_01 [DD68E35AF41A4CA6AACCF6599A70F5E2]
2016-01-14 17:16:23,720 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 29904394FEEA402EB30715E427A84ABF on vertex vertex_1452791783520_0001_1_00 [D5BAC4FF61A8486F8898DD3401238B1C]
2016-01-14 17:16:23,725 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=2F2A1170585046319A32B83434535562, vertexName=DD68E35AF41A4CA6AACCF6599A70F5E2, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:23,726 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=2F2A1170585046319A32B83434535562, vertexId=vertex_1452791783520_0001_1_01 [DD68E35AF41A4CA6AACCF6599A70F5E2]
2016-01-14 17:16:23,726 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:23,727 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 29904394FEEA402EB30715E427A84ABF asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:23,727 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:23,728 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:23,729 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:23,729 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791783520_0001_1][Event:VERTEX_INITIALIZED]: vertexName=DD68E35AF41A4CA6AACCF6599A70F5E2, vertexId=vertex_1452791783520_0001_1_01, initRequestedTime=1452791783715, initedTime=1452791783729, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:23,729 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791783520_0001_1_01 [DD68E35AF41A4CA6AACCF6599A70F5E2] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:23,731 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:23,732 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 29904394FEEA402EB30715E427A84ABF on vertex vertex_1452791783520_0001_1_00 [D5BAC4FF61A8486F8898DD3401238B1C]
2016-01-14 17:16:23,732 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791783520_0001_1_00 [D5BAC4FF61A8486F8898DD3401238B1C] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:23,732 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {29904394FEEA402EB30715E427A84ABF=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:23,733 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791783520_0001_1_00 [D5BAC4FF61A8486F8898DD3401238B1C] parallelism set to 1
2016-01-14 17:16:23,733 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791783520_0001_1_00 [D5BAC4FF61A8486F8898DD3401238B1C] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:23,734 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791783520_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791783520_0001_1_00, eventCount=1
2016-01-14 17:16:23,734 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791783520_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D5BAC4FF61A8486F8898DD3401238B1C, vertexId=vertex_1452791783520_0001_1_00, initRequestedTime=1452791783713, initedTime=1452791783734, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:23,734 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:23,735 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791783520_0001_1_00 [D5BAC4FF61A8486F8898DD3401238B1C] with distanceFromRoot: 0
2016-01-14 17:16:23,735 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791783520_0001_1_00 [D5BAC4FF61A8486F8898DD3401238B1C] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:23,736 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D5BAC4FF61A8486F8898DD3401238B1C
2016-01-14 17:16:23,736 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791783520_0001_1_00 [D5BAC4FF61A8486F8898DD3401238B1C]
2016-01-14 17:16:23,736 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791783520_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791783520_0001_1_00, startRequestedTime=1452791783736, startedTime=1452791783736
2016-01-14 17:16:23,736 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:23,737 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791783520_0001_1_00 [D5BAC4FF61A8486F8898DD3401238B1C] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:23,738 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:23,738 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791783520_0001_1][Event:TASK_STARTED]: vertexName=D5BAC4FF61A8486F8898DD3401238B1C, taskId=task_1452791783520_0001_1_00_000000, scheduledTime=1452791783737, launchTime=1452791783737
2016-01-14 17:16:23,738 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791783520_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:23,739 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791783520_0001_1_00 for vertex: vertex_1452791783520_0001_1_01 [DD68E35AF41A4CA6AACCF6599A70F5E2] numStartedSources: 1 numSources: 1
2016-01-14 17:16:23,739 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791783520_0001_1_01 [DD68E35AF41A4CA6AACCF6599A70F5E2] with distanceFromRoot: 1
2016-01-14 17:16:23,739 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791783520_0001_1_00 [D5BAC4FF61A8486F8898DD3401238B1C]
2016-01-14 17:16:23,740 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: D5BAC4FF61A8486F8898DD3401238B1C in vertex: DD68E35AF41A4CA6AACCF6599A70F5E2
2016-01-14 17:16:23,740 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: DD68E35AF41A4CA6AACCF6599A70F5E2 with 1 source tasks and 1 pending tasks
2016-01-14 17:16:23,740 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791783520_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791783520_0001_1_01, startRequestedTime=1452791783740, startedTime=1452791783740
2016-01-14 17:16:23,740 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:23,741 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791783520_0001_1_01 [DD68E35AF41A4CA6AACCF6599A70F5E2] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:23,741 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [63CD34C4A54D416CA144CEBF2B5B0049/DF79026A731B48D3979A867A7AD3E43B] every/(1/1) ...blyplatform/-1_UNKNOWN_-1, VertexName: D5BAC4FF61A8486F8898DD3401238B1C, VertexParallelism: 1, TaskAttemptID:attempt_1452791783520_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=29904394FEEA402EB30715E427A84ABF, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=DD68E35AF41A4CA6AACCF6599A70F5E2, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:23,742 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791783520_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:23,742 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:23,743 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:23,743 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:23,744 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:23,744 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791783520_0001_00_000001
2016-01-14 17:16:23,744 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791783520_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:23,746 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791783520_0001_00_000001
2016-01-14 17:16:23,746 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:23,747 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791783520_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:23,747 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:23,748 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791783520_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791783520_0001_00_000001, launchTime=1452791783747
2016-01-14 17:16:23,748 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791783520_0001_00_000001
2016-01-14 17:16:23,749 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791783520_0001_1_00_000000_0] to container: [container_1452791783520_0001_00_000001]
2016-01-14 17:16:23,749 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791783520_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:23,749 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791783520_0001_00_000001 given task: attempt_1452791783520_0001_1_00_000000_0
2016-01-14 17:16:23,750 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791783520_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791783520_0001_1_00_000000_0
2016-01-14 17:16:23,750 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791783520_0001_1_00_000000_0] started. Is using containerId: [container_1452791783520_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:23,750 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791783520_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D5BAC4FF61A8486F8898DD3401238B1C, taskAttemptId=attempt_1452791783520_0001_1_00_000000_0, startTime=1452791783749, containerId=container_1452791783520_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791783520_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:23,750 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:23,751 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791783520_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:23,751 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791783520_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:23 Starting to run new task attempt: attempt_1452791783520_0001_1_00_000000_0
2016-01-14 17:16:23,751 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791783520_0001_1_00_000000_0
2016-01-14 17:16:23,751 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:23,752 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:23,752 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:23,752 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:23,752 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:23,752 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:23,752 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [63CD34C4A54D416CA144CEBF2B5B0049/DF79026A731B48D3979A867A7AD3E43B] every/(1/1) ...blyplatform/-1_UNKNOWN_-1, VertexName: D5BAC4FF61A8486F8898DD3401238B1C, VertexParallelism: 1, TaskAttemptID:attempt_1452791783520_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=29904394FEEA402EB30715E427A84ABF, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=DD68E35AF41A4CA6AACCF6599A70F5E2, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:23,753 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:16:23,756 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791783520_0001, with dag id: dag_1452791783520_0001_1
2016-01-14 17:16:23,756 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791783520_0001_1_00_000000_0
2016-01-14 17:16:23,757 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:23,757 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:23,759 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=29904394FEEA402EB30715E427A84ABF, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:23,760 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:23,760 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 29904394FEEA402EB30715E427A84ABF
2016-01-14 17:16:23,761 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 29904394FEEA402EB30715E427A84ABF
2016-01-14 17:16:23,763 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:23,767 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:23,771 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=DD68E35AF41A4CA6AACCF6599A70F5E2, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:23,773 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:23,774 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: DD68E35AF41A4CA6AACCF6599A70F5E2
2016-01-14 17:16:23,776 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:23,776 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: DD68E35AF41A4CA6AACCF6599A70F5E2
2016-01-14 17:16:23,771 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:23,772 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791783520_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:23,777 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 29904394FEEA402EB30715E427A84ABF
2016-01-14 17:16:23,777 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 29904394FEEA402EB30715E427A84ABF
2016-01-14 17:16:23,777 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: DD68E35AF41A4CA6AACCF6599A70F5E2
2016-01-14 17:16:23,780 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D5BAC4FF61A8486F8898DD3401238B1C, ordinal: 0
2016-01-14 17:16:23,780 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D5BAC4FF61A8486F8898DD3401238B1C, mem on start (mb), free: 237, total: 490, max: 672
2016-01-14 17:16:23,780 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:23,781 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:23,781 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:23,781 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:23,781 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:23,781 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:23,781 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:23,781 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:23,782 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:23,782 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 17:16:23,782 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 29904394FEEA402EB30715E427A84ABF, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:23,783 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, DD68E35AF41A4CA6AACCF6599A70F5E2, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:23,783 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:23,784 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 29904394FEEA402EB30715E427A84ABF
2016-01-14 17:16:23,784 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:23,784 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 29904394FEEA402EB30715E427A84ABF
2016-01-14 17:16:23,784 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 29904394FEEA402EB30715E427A84ABF being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:23,785 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:23,786 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:23,786 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:23,786 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791783520_0001_1_00_000000_0
2016-01-14 17:16:23,790 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:23,791 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 29904394FEEA402EB30715E427A84ABF
2016-01-14 17:16:23,791 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 5ACEEA3CB2554523BE3A761C28D888EC
2016-01-14 17:16:23,791 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 5ACEEA3CB2554523BE3A761C28D888EC
2016-01-14 17:16:23,792 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@4e7f1fc4; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@7bfea51e; comparator=cascading.tuple.hadoop.util.TupleComparator@b0a942b; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:23,794 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:23,794 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:23,795 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:23,795 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:23,795 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@57ed07c5
2016-01-14 17:16:23,799 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@57ed07c5
2016-01-14 17:16:23,799 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:23,799 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:23,812 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:23,812 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:23,812 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:23,812 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:23,813 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:23,813 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:23,813 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:23,814 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D5BAC4FF61A8486F8898DD3401238B1C, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:23,816 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791783520_0001_1_00_000000_0
2016-01-14 17:16:23,817 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D5BAC4FF61A8486F8898DD3401238B1C, mem on close (mb), free: 135, total: 490, max: 672
2016-01-14 17:16:23,817 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:23,817 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:23,817 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:23,817 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:23,820 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:23 Completed running task attempt: attempt_1452791783520_0001_1_00_000000_0
2016-01-14 17:16:23,825 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791783520_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:23,826 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=513802240, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791783814, Process_Duration=2, Process_End_Time=1452791783816, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:23,826 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791783520_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791783520_0001_1_00, eventCount=1
2016-01-14 17:16:23,826 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:23,827 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:23,827 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791783520_0001_00_000001
2016-01-14 17:16:23,827 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:23,827 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791783520_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D5BAC4FF61A8486F8898DD3401238B1C, taskAttemptId=attempt_1452791783520_0001_1_00_000000_0, startTime=1452791783749, finishTime=1452791783827, timeTaken=78, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=513802240, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791783814, Process_Duration=2, Process_End_Time=1452791783816, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:23,828 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791783520_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:23,828 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791783520_0001_1_00_000000_0
2016-01-14 17:16:23,829 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:23,829 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791783520_0001_1][Event:TASK_FINISHED]: vertexName=D5BAC4FF61A8486F8898DD3401238B1C, taskId=task_1452791783520_0001_1_00_000000, startTime=1452791783749, finishTime=1452791783828, timeTaken=79, status=SUCCEEDED, successfulAttemptID=attempt_1452791783520_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=513802240, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791783814, Process_Duration=2, Process_End_Time=1452791783816, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:23,829 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791783520_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:23,829 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791783520_0001_1_00 [D5BAC4FF61A8486F8898DD3401238B1C] : 1
2016-01-14 17:16:23,829 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791783520_0001_1_00 [D5BAC4FF61A8486F8898DD3401238B1C]
2016-01-14 17:16:23,830 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:23,831 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:23,831 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791783520_0001_1][Event:VERTEX_FINISHED]: vertexName=D5BAC4FF61A8486F8898DD3401238B1C, vertexId=vertex_1452791783520_0001_1_00, initRequestedTime=1452791783713, initedTime=1452791783734, startRequestedTime=1452791783736, startedTime=1452791783736, finishTime=1452791783829, timeTaken=93, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=513802240, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791783814, Process_Duration=2, Process_End_Time=1452791783816, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791783749, firstTasksToStart=[ task_1452791783520_0001_1_00_000000 ], lastTaskFinishTime=1452791783827, lastTasksToFinish=[ task_1452791783520_0001_1_00_000000 ], minTaskDuration=78, maxTaskDuration=78, avgTaskDuration=78.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791783520_0001_1_00_000000 ], longestDurationTasks=[ task_1452791783520_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:23,832 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791783520_0001_1_00 [D5BAC4FF61A8486F8898DD3401238B1C] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:23,832 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791783520_0001_1_01 [DD68E35AF41A4CA6AACCF6599A70F5E2] attempt: attempt_1452791783520_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:23,832 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: DD68E35AF41A4CA6AACCF6599A70F5E2
2016-01-14 17:16:23,832 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791783520_0001_1_01 [DD68E35AF41A4CA6AACCF6599A70F5E2]
2016-01-14 17:16:23,832 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791783520_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:23,833 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791783520_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791783520_0001_00_000001, stoppedTime=1452791783833, exitStatus=0
2016-01-14 17:16:23,833 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:23,833 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791783520_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:23,833 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791783520_0001_1_00 [D5BAC4FF61A8486F8898DD3401238B1C] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:23,834 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791783520_0001_1][Event:TASK_STARTED]: vertexName=DD68E35AF41A4CA6AACCF6599A70F5E2, taskId=task_1452791783520_0001_1_01_000000, scheduledTime=1452791783834, launchTime=1452791783834
2016-01-14 17:16:23,834 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:23,834 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791783520_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:23,835 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791783520_0001_1_01 [DD68E35AF41A4CA6AACCF6599A70F5E2]
2016-01-14 17:16:23,835 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791783520_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:23,836 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [63CD34C4A54D416CA144CEBF2B5B0049/DF79026A731B48D3979A867A7AD3E43B] every/(1/1) ...blyplatform/-1_UNKNOWN_-1, VertexName: DD68E35AF41A4CA6AACCF6599A70F5E2, VertexParallelism: 1, TaskAttemptID:attempt_1452791783520_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D5BAC4FF61A8486F8898DD3401238B1C, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=2F2A1170585046319A32B83434535562, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:23,837 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791783520_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:23,837 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791783520_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:23,838 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:23,838 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791783520_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:23,839 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791783520_0001_00_000001: 12 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:23,838 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791783520_0001_00_000002
2016-01-14 17:16:23,839 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791783520_0001_00_000001, Exiting
2016-01-14 17:16:23,840 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791783520_0001_00_000001
2016-01-14 17:16:23,840 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791783520_0001_00_000001
2016-01-14 17:16:23,839 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791783520_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:23,841 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791783520_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:23,842 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791783520_0001_00_000001 completed successfully
2016-01-14 17:16:23,843 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791783520_0001_00_000002
2016-01-14 17:16:23,844 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 17:16:23,847 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791783520_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:23,848 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791783520_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:23,849 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791783520_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791783520_0001_00_000002, launchTime=1452791783848
2016-01-14 17:16:23,849 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:23,850 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791783520_0001_00_000002
2016-01-14 17:16:23,851 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791783520_0001_1_01_000000_0] to container: [container_1452791783520_0001_00_000002]
2016-01-14 17:16:23,851 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791783520_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:23,851 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791783520_0001_00_000002 given task: attempt_1452791783520_0001_1_01_000000_0
2016-01-14 17:16:23,851 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791783520_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791783520_0001_1_01_000000_0
2016-01-14 17:16:23 Starting to run new task attempt: attempt_1452791783520_0001_1_01_000000_0
2016-01-14 17:16:23,852 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791783520_0001_1_01_000000_0] started. Is using containerId: [container_1452791783520_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:23,852 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791783520_0001_1_01_000000_0
2016-01-14 17:16:23,852 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:23,852 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:23,853 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:23,853 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791783520_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=DD68E35AF41A4CA6AACCF6599A70F5E2, taskAttemptId=attempt_1452791783520_0001_1_01_000000_0, startTime=1452791783852, containerId=container_1452791783520_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791783520_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:23,853 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:23,853 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791783520_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:23,853 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:23,853 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791783520_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:23,853 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:23,854 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:23,854 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [63CD34C4A54D416CA144CEBF2B5B0049/DF79026A731B48D3979A867A7AD3E43B] every/(1/1) ...blyplatform/-1_UNKNOWN_-1, VertexName: DD68E35AF41A4CA6AACCF6599A70F5E2, VertexParallelism: 1, TaskAttemptID:attempt_1452791783520_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D5BAC4FF61A8486F8898DD3401238B1C, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=2F2A1170585046319A32B83434535562, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:23,854 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:16:23,855 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791783520_0001_1_01_000000_0
2016-01-14 17:16:23,855 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:23,855 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:23,856 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:23,856 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=D5BAC4FF61A8486F8898DD3401238B1C, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:23,857 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:23,857 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: D5BAC4FF61A8486F8898DD3401238B1C
2016-01-14 17:16:23,857 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791783520_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:23,859 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:23,859 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:23,859 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=2F2A1170585046319A32B83434535562, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:23,859 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:23,859 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: D5BAC4FF61A8486F8898DD3401238B1C
2016-01-14 17:16:23,864 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: D5BAC4FF61A8486F8898DD3401238B1C
2016-01-14 17:16:23,864 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:23,864 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 2F2A1170585046319A32B83434535562
2016-01-14 17:16:23,865 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:23,865 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 2F2A1170585046319A32B83434535562
2016-01-14 17:16:23,869 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: DD68E35AF41A4CA6AACCF6599A70F5E2, ordinal: 1
2016-01-14 17:16:23,869 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DD68E35AF41A4CA6AACCF6599A70F5E2, mem on start (mb), free: 128, total: 490, max: 672
2016-01-14 17:16:23,869 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:23,869 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:23,869 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:23,875 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:23,875 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 2F2A1170585046319A32B83434535562
2016-01-14 17:16:23,876 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:23,876 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:23,876 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:23,876 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:23,876 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:23,877 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:23,877 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:23,877 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, D5BAC4FF61A8486F8898DD3401238B1C, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 17:16:23,877 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 2F2A1170585046319A32B83434535562, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:23,877 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:23,877 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: D5BAC4FF61A8486F8898DD3401238B1C being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:23,877 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: D5BAC4FF61A8486F8898DD3401238B1C
2016-01-14 17:16:23,877 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:23,878 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:23,879 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:23,879 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:23,879 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: D5BAC4FF61A8486F8898DD3401238B1C with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:23,879 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 17:16:23,880 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:23,880 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:23,880 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: D5BAC4FF61A8486F8898DD3401238B1C
2016-01-14 17:16:23,881 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:23,884 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791783520_0001_1_01_000000_0
2016-01-14 17:16:23,884 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791783520_0001_1_00_000000_0_10033, runDuration: 49369, ]
2016-01-14 17:16:23,886 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791783520_0001_1_00_000000_0_10033] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791783886, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 17:16:23,887 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:23,887 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : D5BAC4FF61A8486F8898DD3401238B1C
2016-01-14 17:16:23,888 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 5ACEEA3CB2554523BE3A761C28D888EC
2016-01-14 17:16:23,888 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_-1"], id: 2F2A1170585046319A32B83434535562
2016-01-14 17:16:23,888 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_-1"]
2016-01-14 17:16:23,889 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 5ACEEA3CB2554523BE3A761C28D888EC, for 1 inputs
2016-01-14 17:16:23,888 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:23,889 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [D5BAC4FF61A8486F8898DD3401238B1C] #6 in 4ms
2016-01-14 17:16:23,890 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [D5BAC4FF61A8486F8898DD3401238B1C] #6, status:false, isInterrupted:false
2016-01-14 17:16:23,890 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:23,890 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:23,891 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:23,891 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:23,891 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:23,891 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : D5BAC4FF61A8486F8898DD3401238B1C
2016-01-14 17:16:23,891 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: DD68E35AF41A4CA6AACCF6599A70F5E2, all 1 inputs ready in: 00:00:00.002
2016-01-14 17:16:23,892 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:23,892 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@36bc2602; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@7a636960
2016-01-14 17:16:23,896 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:23,896 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:23,896 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791783520_0001_1_01_000000_0
2016-01-14 17:16:23,897 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791783520_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:23,897 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791783520_0001_1_01_000000_0
2016-01-14 17:16:23,897 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DD68E35AF41A4CA6AACCF6599A70F5E2, mem on close (mb), free: 125, total: 490, max: 672
2016-01-14 17:16:23,898 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: D5BAC4FF61A8486F8898DD3401238B1C
2016-01-14 17:16:23,899 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:23,899 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:23,900 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:23,900 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:23,900 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:23 Completed running task attempt: attempt_1452791783520_0001_1_01_000000_0
2016-01-14 17:16:23,901 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791783520_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:23,902 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:23,902 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791783520_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=DD68E35AF41A4CA6AACCF6599A70F5E2, taskAttemptId=attempt_1452791783520_0001_1_01_000000_0, startTime=1452791783852, finishTime=1452791783902, timeTaken=50, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=285, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=513802240, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791783892, Process_Duration=5, Process_End_Time=1452791783897, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:23,902 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791783520_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:23,902 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791783520_0001_1_01_000000_0
2016-01-14 17:16:23,903 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:23,903 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791783520_0001_1][Event:TASK_FINISHED]: vertexName=DD68E35AF41A4CA6AACCF6599A70F5E2, taskId=task_1452791783520_0001_1_01_000000, startTime=1452791783852, finishTime=1452791783903, timeTaken=51, status=SUCCEEDED, successfulAttemptID=attempt_1452791783520_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=285, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=513802240, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791783892, Process_Duration=5, Process_End_Time=1452791783897, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:23,903 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:23,903 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791783520_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:23,904 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791783520_0001_1_01 [DD68E35AF41A4CA6AACCF6599A70F5E2] : 1
2016-01-14 17:16:23,904 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791783520_0001_1_01 [DD68E35AF41A4CA6AACCF6599A70F5E2]
2016-01-14 17:16:23,905 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:23,905 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791783520_0001_1][Event:VERTEX_FINISHED]: vertexName=DD68E35AF41A4CA6AACCF6599A70F5E2, vertexId=vertex_1452791783520_0001_1_01, initRequestedTime=1452791783715, initedTime=1452791783729, startRequestedTime=1452791783740, startedTime=1452791783740, finishTime=1452791783904, timeTaken=164, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=285, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=513802240, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791783892, Process_Duration=5, Process_End_Time=1452791783897, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452791783852, firstTasksToStart=[ task_1452791783520_0001_1_01_000000 ], lastTaskFinishTime=1452791783902, lastTasksToFinish=[ task_1452791783520_0001_1_01_000000 ], minTaskDuration=50, maxTaskDuration=50, avgTaskDuration=50.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791783520_0001_1_01_000000 ], longestDurationTasks=[ task_1452791783520_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:23,905 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791783520_0001_1_01 [DD68E35AF41A4CA6AACCF6599A70F5E2] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:23,906 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=285, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=513802240, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791783892, Process_Duration=5, Process_End_Time=1452791783897, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 17:16:23,906 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:23,906 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791783520_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:23,907 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:23,907 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791783520_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791783520_0001_00_000002, stoppedTime=1452791783907, exitStatus=0
2016-01-14 17:16:23,907 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:23,907 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791783520_0001_00_000002
2016-01-14 17:16:23,907 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791783520_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 17:16:23,907 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791783520_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:23,907 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791783520_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:23,907 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791783520_0001_00_000002, Exiting
2016-01-14 17:16:23,908 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791783520_0001_1_01 [DD68E35AF41A4CA6AACCF6599A70F5E2] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:23,908 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791783520_0001_00_000002
2016-01-14 17:16:23,908 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791783520_0001_00_000002
2016-01-14 17:16:23,908 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791783520_0001_1
2016-01-14 17:16:23,909 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791783520_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452791783520_0001_1
2016-01-14 17:16:23,909 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791783520_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:23,909 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791783520_0001_00_000002 completed successfully
2016-01-14 17:16:23,909 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 2F2A1170585046319A32B83434535562 for vertex: vertex_1452791783520_0001_1_01 [DD68E35AF41A4CA6AACCF6599A70F5E2]
2016-01-14 17:16:23,919 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452791783520_0001_1_00 [D5BAC4FF61A8486F8898DD3401238B1C]
2016-01-14 17:16:23,921 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791783520_0001_1, queueSize=0
2016-01-14 17:16:23,921 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:23,921 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791783520_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791783520_0001_1, startTime=1452791783712, finishTime=1452791783919, timeTaken=207, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=618, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1027604480, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905583567706, Process_Duration=7, Process_End_Time=2905583567713, Read_Duration=1, Tuples_Read=40, Tuples_Written=40, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 17:16:23,921 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791783520_0001_1 finished with state: SUCCEEDED
2016-01-14 17:16:23,922 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791783520_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:23,922 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452791783520_0001_00_000002
2016-01-14 17:16:23,922 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791783520_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:16:23 Completed Dag: dag_1452791783520_0001_1
2016-01-14 17:16:23,924 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791783520_0001_1, dagState=SUCCEEDED
2016-01-14 17:16:23,924 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791783520_0001_1_post
2016-01-14 17:16:23,925 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:23,925 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:23,930 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...blyplatform/-1_UNKNOWN_-1, applicationId=application_1452791783520_0001
2016-01-14 17:16:23,932 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:23,932 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:23,932 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:23,932 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:23,932 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:23,933 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:23,933 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:23,933 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:23,933 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:23,934 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:23,934 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:23,934 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:23,934 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:23,939 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/63CD34C4A54D416CA144CEBF2B5B0049/DF79026A731B48D3979A867A7AD3E43B/.tez/application_1452791783520_0001
2016-01-14 17:16:23,940 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.446
2016-01-14 17:16:24,022 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_UNKNOWN_-1
2016-01-14 17:16:24,052 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:24,053 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:24,087 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.008
2016-01-14 17:16:24,093 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.014
2016-01-14 17:16:24,094 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 17:16:24,094 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 17:16:24,094 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_UNKNOWN_'field'
2016-01-14 17:16:24,120 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:24,120 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:24,168 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.020
2016-01-14 17:16:24,178 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.030
2016-01-14 17:16:24,179 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:24,179 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:24,179 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:24,196 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:24,203 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_RESULTS
2016-01-14 17:16:24,259 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:24,260 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:24,261 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_RESULTS"]
2016-01-14 17:16:24,261 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:24,261 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:24,262 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:24,263 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...atform/-1_UNKNOWN_RESULTS
2016-01-14 17:16:24,263 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:24,276 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:24,281 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:24,281 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:24,281 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:24,291 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/DBD5D3CB4C6B4869B80820C5DA0FFE69/F5B5C0E4F3364F9187441605FF7F2311/.tez/application_1452791784281_0001 doesn't exist and is created
2016-01-14 17:16:24,310 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:24,310 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:24,310 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/DBD5D3CB4C6B4869B80820C5DA0FFE69/F5B5C0E4F3364F9187441605FF7F2311/.tez/application_1452791784281_0001
2016-01-14 17:16:24,314 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791784281_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:24,339 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:24,341 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:24,342 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:24,343 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:24,343 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/DBD5D3CB4C6B4869B80820C5DA0FFE69/F5B5C0E4F3364F9187441605FF7F2311/.tez/application_1452791784281_0001/localmode-log-dir
2016-01-14 17:16:24,343 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/DBD5D3CB4C6B4869B80820C5DA0FFE69/F5B5C0E4F3364F9187441605FF7F2311/.tez/application_1452791784281_0001/localmode-log-dir/history.txt.appattempt_1452791784281_0001_000000, maxErrors=10
2016-01-14 17:16:24,343 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:24,344 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791784281_0001_000000, appSubmitTime=1452791784314, launchTime=1452791784314
2016-01-14 17:16:24,347 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:24,348 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:24,351 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:24,352 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:24,353 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:54261
2016-01-14 17:16:24,356 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:24,358 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791784281_0001_000000, startTime=1452791784358
2016-01-14 17:16:24,358 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:24,358 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:24,410 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:24,410 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:24,410 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...atform/-1_UNKNOWN_RESULTS, applicationId=application_1452791784281_0001, dagName=[DBD5D3CB4C6B4869B80820C5DA0FFE69/F5B5C0E4F3364F9187441605FF7F2311] every/(1/1) ...atform/-1_UNKNOWN_RESULTS
2016-01-14 17:16:24,428 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [DBD5D3CB4C6B4869B80820C5DA0FFE69/F5B5C0E4F3364F9187441605FF7F2311] every/(1/1) ...atform/-1_UNKNOWN_RESULTS
2016-01-14 17:16:24,429 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791784281_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/DBD5D3CB4C6B4869B80820C5DA0FFE69/F5B5C0E4F3364F9187441605FF7F2311/.tez/application_1452791784281_0001/localmode-log-dir/dag_1452791784281_0001_1.dot
2016-01-14 17:16:24,430 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/DBD5D3CB4C6B4869B80820C5DA0FFE69/F5B5C0E4F3364F9187441605FF7F2311/.tez/application_1452791784281_0001/localmode-log-dir/dag_1452791784281_0001_1-tez-dag.pb.txt
2016-01-14 17:16:24,439 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791784281_0001_1
2016-01-14 17:16:24,439 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:24,439 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [DBD5D3CB4C6B4869B80820C5DA0FFE69/F5B5C0E4F3364F9187441605FF7F2311] every/(1/1) ...atform/-1_UNKNOWN_RESULTS
2016-01-14 17:16:24 Running Dag: dag_1452791784281_0001_1
2016-01-14 17:16:24,452 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791784281_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791784281_0001_1, submitTime=1452791784428
2016-01-14 17:16:24,452 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:24,453 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex F88F5CCAA8AA43569D44AF583AC1357D
2016-01-14 17:16:24,453 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:24,454 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791784281_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791784281_0001_1, initTime=1452791784452
2016-01-14 17:16:24,454 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:24,455 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791784281_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:24,456 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791784281_0001_1][Event:DAG_STARTED]: dagID=dag_1452791784281_0001_1, startTime=1452791784455
2016-01-14 17:16:24,456 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:24,456 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:24,456 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791784281_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:24,456 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: A1526F8B76874BB384D1F52C001AB8BA : {7C62D370ADDC4855B1F3844F092B0732={InputName=7C62D370ADDC4855B1F3844F092B0732}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:24,456 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 7C62D370ADDC4855B1F3844F092B0732, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:24,457 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791784281_0001_1_00 [A1526F8B76874BB384D1F52C001AB8BA]
2016-01-14 17:16:24,457 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791784281_0001_1_00 [A1526F8B76874BB384D1F52C001AB8BA]
2016-01-14 17:16:24,457 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791784281_0001_1_00 [A1526F8B76874BB384D1F52C001AB8BA]
2016-01-14 17:16:24,457 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791784281_0001_1_00 [A1526F8B76874BB384D1F52C001AB8BA]. Starting root input initializers: 1
2016-01-14 17:16:24,458 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...atform/-1_UNKNOWN_RESULTS, applicationId=application_1452791784281_0001, dagName=[DBD5D3CB4C6B4869B80820C5DA0FFE69/F5B5C0E4F3364F9187441605FF7F2311] every/(1/1) ...atform/-1_UNKNOWN_RESULTS
2016-01-14 17:16:24,458 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791784281_0001_1_00 [A1526F8B76874BB384D1F52C001AB8BA] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:24,458 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 7C62D370ADDC4855B1F3844F092B0732 on vertex vertex_1452791784281_0001_1_00 [A1526F8B76874BB384D1F52C001AB8BA]
2016-01-14 17:16:24,458 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791784281_0001_1_01 [F88F5CCAA8AA43569D44AF583AC1357D]
2016-01-14 17:16:24,460 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:24,460 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 7C62D370ADDC4855B1F3844F092B0732 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:24,461 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:24,461 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:24,462 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:24,462 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791784281_0001_1_01 [F88F5CCAA8AA43569D44AF583AC1357D]
2016-01-14 17:16:24,462 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791784281_0001_1_01 [F88F5CCAA8AA43569D44AF583AC1357D]
2016-01-14 17:16:24,462 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791784281_0001_1_01 [F88F5CCAA8AA43569D44AF583AC1357D]
2016-01-14 17:16:24,462 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=E88468F2E09048D19120F6ABAC84E963, vertexId=vertex_1452791784281_0001_1_01 [F88F5CCAA8AA43569D44AF583AC1357D], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:24,463 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=E88468F2E09048D19120F6ABAC84E963, vertexId=vertex_1452791784281_0001_1_01 [F88F5CCAA8AA43569D44AF583AC1357D]
2016-01-14 17:16:24,464 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:24,465 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 7C62D370ADDC4855B1F3844F092B0732 on vertex vertex_1452791784281_0001_1_00 [A1526F8B76874BB384D1F52C001AB8BA]
2016-01-14 17:16:24,465 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=E88468F2E09048D19120F6ABAC84E963, vertexName=F88F5CCAA8AA43569D44AF583AC1357D, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:24,466 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=E88468F2E09048D19120F6ABAC84E963, vertexId=vertex_1452791784281_0001_1_01 [F88F5CCAA8AA43569D44AF583AC1357D]
2016-01-14 17:16:24,467 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:24,467 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791784281_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F88F5CCAA8AA43569D44AF583AC1357D, vertexId=vertex_1452791784281_0001_1_01, initRequestedTime=1452791784458, initedTime=1452791784467, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:24,468 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791784281_0001_1_01 [F88F5CCAA8AA43569D44AF583AC1357D] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:24,468 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791784281_0001_1_00 [A1526F8B76874BB384D1F52C001AB8BA] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:24,468 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {7C62D370ADDC4855B1F3844F092B0732=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:24,468 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791784281_0001_1_00 [A1526F8B76874BB384D1F52C001AB8BA] parallelism set to 1
2016-01-14 17:16:24,468 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791784281_0001_1_00 [A1526F8B76874BB384D1F52C001AB8BA] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:24,469 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791784281_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791784281_0001_1_00, eventCount=1
2016-01-14 17:16:24,470 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:24,470 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791784281_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A1526F8B76874BB384D1F52C001AB8BA, vertexId=vertex_1452791784281_0001_1_00, initRequestedTime=1452791784456, initedTime=1452791784469, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:24,470 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791784281_0001_1_00 [A1526F8B76874BB384D1F52C001AB8BA] with distanceFromRoot: 0
2016-01-14 17:16:24,470 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791784281_0001_1_00 [A1526F8B76874BB384D1F52C001AB8BA] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:24,470 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in A1526F8B76874BB384D1F52C001AB8BA
2016-01-14 17:16:24,471 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791784281_0001_1_00 [A1526F8B76874BB384D1F52C001AB8BA]
2016-01-14 17:16:24,471 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:24,472 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791784281_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791784281_0001_1_00, startRequestedTime=1452791784470, startedTime=1452791784470
2016-01-14 17:16:24,472 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791784281_0001_1_00 [A1526F8B76874BB384D1F52C001AB8BA] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:24,473 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:24,473 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791784281_0001_1][Event:TASK_STARTED]: vertexName=A1526F8B76874BB384D1F52C001AB8BA, taskId=task_1452791784281_0001_1_00_000000, scheduledTime=1452791784472, launchTime=1452791784472
2016-01-14 17:16:24,473 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791784281_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:24,474 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791784281_0001_1_00 for vertex: vertex_1452791784281_0001_1_01 [F88F5CCAA8AA43569D44AF583AC1357D] numStartedSources: 1 numSources: 1
2016-01-14 17:16:24,474 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791784281_0001_1_01 [F88F5CCAA8AA43569D44AF583AC1357D] with distanceFromRoot: 1
2016-01-14 17:16:24,474 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791784281_0001_1_00 [A1526F8B76874BB384D1F52C001AB8BA]
2016-01-14 17:16:24,475 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: A1526F8B76874BB384D1F52C001AB8BA in vertex: F88F5CCAA8AA43569D44AF583AC1357D
2016-01-14 17:16:24,475 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: F88F5CCAA8AA43569D44AF583AC1357D with 1 source tasks and 1 pending tasks
2016-01-14 17:16:24,476 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:24,476 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791784281_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791784281_0001_1_01, startRequestedTime=1452791784475, startedTime=1452791784475
2016-01-14 17:16:24,476 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791784281_0001_1_01 [F88F5CCAA8AA43569D44AF583AC1357D] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:24,477 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [DBD5D3CB4C6B4869B80820C5DA0FFE69/F5B5C0E4F3364F9187441605FF7F2311] every/(1/1) ...atform/-1_UNKNOWN_RESULTS, VertexName: A1526F8B76874BB384D1F52C001AB8BA, VertexParallelism: 1, TaskAttemptID:attempt_1452791784281_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=7C62D370ADDC4855B1F3844F092B0732, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F88F5CCAA8AA43569D44AF583AC1357D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:24,477 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791784281_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:24,477 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:24,478 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:24,478 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:24,479 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:24,479 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791784281_0001_00_000001
2016-01-14 17:16:24,479 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791784281_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:24,481 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791784281_0001_00_000001
2016-01-14 17:16:24,481 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:24,482 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791784281_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:24,482 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:24,483 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791784281_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791784281_0001_00_000001, launchTime=1452791784482
2016-01-14 17:16:24,483 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791784281_0001_00_000001
2016-01-14 17:16:24,485 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791784281_0001_1_00_000000_0] to container: [container_1452791784281_0001_00_000001]
2016-01-14 17:16:24,485 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791784281_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:24,486 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791784281_0001_1_00_000000_0] started. Is using containerId: [container_1452791784281_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:24,486 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791784281_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A1526F8B76874BB384D1F52C001AB8BA, taskAttemptId=attempt_1452791784281_0001_1_00_000000_0, startTime=1452791784486, containerId=container_1452791784281_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791784281_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:24,486 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791784281_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:24,486 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:24,486 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791784281_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:24,487 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791784281_0001_00_000001 given task: attempt_1452791784281_0001_1_00_000000_0
2016-01-14 17:16:24,487 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791784281_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791784281_0001_1_00_000000_0
2016-01-14 17:16:24 Starting to run new task attempt: attempt_1452791784281_0001_1_00_000000_0
2016-01-14 17:16:24,487 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791784281_0001_1_00_000000_0
2016-01-14 17:16:24,487 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:24,488 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:24,488 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:24,488 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:24,488 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:24,488 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:24,488 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [DBD5D3CB4C6B4869B80820C5DA0FFE69/F5B5C0E4F3364F9187441605FF7F2311] every/(1/1) ...atform/-1_UNKNOWN_RESULTS, VertexName: A1526F8B76874BB384D1F52C001AB8BA, VertexParallelism: 1, TaskAttemptID:attempt_1452791784281_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=7C62D370ADDC4855B1F3844F092B0732, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F88F5CCAA8AA43569D44AF583AC1357D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:24,489 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:16:24,490 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791784281_0001_1_00_000000_0
2016-01-14 17:16:24,490 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:24,490 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:24,491 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=7C62D370ADDC4855B1F3844F092B0732, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:24,492 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:24,492 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 7C62D370ADDC4855B1F3844F092B0732
2016-01-14 17:16:24,494 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 7C62D370ADDC4855B1F3844F092B0732
2016-01-14 17:16:24,493 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F88F5CCAA8AA43569D44AF583AC1357D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:24,495 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791784281_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:24,492 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:24,497 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:24,497 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F88F5CCAA8AA43569D44AF583AC1357D
2016-01-14 17:16:24,500 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:24,501 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:24,501 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: F88F5CCAA8AA43569D44AF583AC1357D
2016-01-14 17:16:24,502 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F88F5CCAA8AA43569D44AF583AC1357D
2016-01-14 17:16:24,507 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791784281_0001, with dag id: dag_1452791784281_0001_1
2016-01-14 17:16:24,508 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A1526F8B76874BB384D1F52C001AB8BA, ordinal: 0
2016-01-14 17:16:24,508 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A1526F8B76874BB384D1F52C001AB8BA, mem on start (mb), free: 230, total: 490, max: 672
2016-01-14 17:16:24,508 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:24,509 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:24,509 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:24,510 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:24,510 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 7C62D370ADDC4855B1F3844F092B0732
2016-01-14 17:16:24,510 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 7C62D370ADDC4855B1F3844F092B0732
2016-01-14 17:16:24,510 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:24,510 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:24,510 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:24,511 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:24,511 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:24,511 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:24,511 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 17:16:24,511 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 7C62D370ADDC4855B1F3844F092B0732, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:24,511 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F88F5CCAA8AA43569D44AF583AC1357D, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:24,511 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:24,512 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 7C62D370ADDC4855B1F3844F092B0732 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:24,512 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 7C62D370ADDC4855B1F3844F092B0732
2016-01-14 17:16:24,512 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:24,513 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 7C62D370ADDC4855B1F3844F092B0732
2016-01-14 17:16:24,514 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:24,514 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:24,514 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:24,515 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791784281_0001_1_00_000000_0
2016-01-14 17:16:24,515 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:24,516 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@7fcc5d6b
2016-01-14 17:16:24,517 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@7fcc5d6b
2016-01-14 17:16:24,517 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:24,518 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:24,519 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:24,519 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 7C62D370ADDC4855B1F3844F092B0732
2016-01-14 17:16:24,520 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: AC463448BFE54A429469C97217F90E84
2016-01-14 17:16:24,520 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] AC463448BFE54A429469C97217F90E84
2016-01-14 17:16:24,521 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@2eed862c; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@6c03cac5; comparator=cascading.tuple.hadoop.util.TupleComparator@6cf582e9; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:24,521 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:24,521 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:24,521 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:24,535 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:24,535 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:24,535 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:24,536 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:24,536 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:24,536 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:24,536 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:24,537 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A1526F8B76874BB384D1F52C001AB8BA, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:24,539 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791784281_0001_1_00_000000_0
2016-01-14 17:16:24,539 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A1526F8B76874BB384D1F52C001AB8BA, mem on close (mb), free: 128, total: 490, max: 672
2016-01-14 17:16:24,539 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:24,539 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:24,540 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:24,540 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:24,542 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:24 Completed running task attempt: attempt_1452791784281_0001_1_00_000000_0
2016-01-14 17:16:24,545 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791784281_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:24,546 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791784537, Process_Duration=2, Process_End_Time=1452791784539, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:24,547 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791784281_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791784281_0001_1_00, eventCount=1
2016-01-14 17:16:24,547 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:24,548 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791784281_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A1526F8B76874BB384D1F52C001AB8BA, taskAttemptId=attempt_1452791784281_0001_1_00_000000_0, startTime=1452791784486, finishTime=1452791784547, timeTaken=61, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791784537, Process_Duration=2, Process_End_Time=1452791784539, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:24,548 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791784281_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:24,548 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791784281_0001_1_00_000000_0
2016-01-14 17:16:24,549 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791784281_0001_1][Event:TASK_FINISHED]: vertexName=A1526F8B76874BB384D1F52C001AB8BA, taskId=task_1452791784281_0001_1_00_000000, startTime=1452791784486, finishTime=1452791784548, timeTaken=62, status=SUCCEEDED, successfulAttemptID=attempt_1452791784281_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791784537, Process_Duration=2, Process_End_Time=1452791784539, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:24,549 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:24,549 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791784281_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:24,549 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791784281_0001_1_00 [A1526F8B76874BB384D1F52C001AB8BA] : 1
2016-01-14 17:16:24,549 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791784281_0001_1_00 [A1526F8B76874BB384D1F52C001AB8BA]
2016-01-14 17:16:24,550 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:24,550 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:24,551 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:24,552 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791784281_0001_00_000001
2016-01-14 17:16:24,552 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:24,552 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791784281_0001_1][Event:VERTEX_FINISHED]: vertexName=A1526F8B76874BB384D1F52C001AB8BA, vertexId=vertex_1452791784281_0001_1_00, initRequestedTime=1452791784456, initedTime=1452791784469, startRequestedTime=1452791784470, startedTime=1452791784470, finishTime=1452791784549, timeTaken=79, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791784537, Process_Duration=2, Process_End_Time=1452791784539, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791784486, firstTasksToStart=[ task_1452791784281_0001_1_00_000000 ], lastTaskFinishTime=1452791784547, lastTasksToFinish=[ task_1452791784281_0001_1_00_000000 ], minTaskDuration=61, maxTaskDuration=61, avgTaskDuration=61.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791784281_0001_1_00_000000 ], longestDurationTasks=[ task_1452791784281_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:24,552 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791784281_0001_1_00 [A1526F8B76874BB384D1F52C001AB8BA] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:24,553 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791784281_0001_1_01 [F88F5CCAA8AA43569D44AF583AC1357D] attempt: attempt_1452791784281_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:24,553 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: F88F5CCAA8AA43569D44AF583AC1357D
2016-01-14 17:16:24,553 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791784281_0001_1_01 [F88F5CCAA8AA43569D44AF583AC1357D]
2016-01-14 17:16:24,554 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791784281_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:24,554 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791784281_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791784281_0001_00_000001, stoppedTime=1452791784554, exitStatus=0
2016-01-14 17:16:24,554 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:24,554 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791784281_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:24,554 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791784281_0001_1_00 [A1526F8B76874BB384D1F52C001AB8BA] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:24,555 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791784281_0001_1][Event:TASK_STARTED]: vertexName=F88F5CCAA8AA43569D44AF583AC1357D, taskId=task_1452791784281_0001_1_01_000000, scheduledTime=1452791784555, launchTime=1452791784555
2016-01-14 17:16:24,555 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:24,555 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791784281_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:24,555 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791784281_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:24,555 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791784281_0001_1_01 [F88F5CCAA8AA43569D44AF583AC1357D]
2016-01-14 17:16:24,556 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791784281_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:24,556 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [DBD5D3CB4C6B4869B80820C5DA0FFE69/F5B5C0E4F3364F9187441605FF7F2311] every/(1/1) ...atform/-1_UNKNOWN_RESULTS, VertexName: F88F5CCAA8AA43569D44AF583AC1357D, VertexParallelism: 1, TaskAttemptID:attempt_1452791784281_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A1526F8B76874BB384D1F52C001AB8BA, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=E88468F2E09048D19120F6ABAC84E963, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:24,556 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791784281_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:24,556 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:24,557 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791784281_0001_00_000002
2016-01-14 17:16:24,558 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791784281_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:24,559 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791784281_0001_00_000002
2016-01-14 17:16:24,559 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:24,560 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791784281_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:24,560 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791784281_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791784281_0001_00_000002, launchTime=1452791784560
2016-01-14 17:16:24,560 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:24,561 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791784281_0001_00_000002
2016-01-14 17:16:24,562 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791784281_0001_1_01_000000_0] to container: [container_1452791784281_0001_00_000002]
2016-01-14 17:16:24,562 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791784281_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:24,563 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791784281_0001_00_000002 given task: attempt_1452791784281_0001_1_01_000000_0
2016-01-14 17:16:24,563 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791784281_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791784281_0001_1_01_000000_0
2016-01-14 17:16:24,563 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791784281_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:24 Starting to run new task attempt: attempt_1452791784281_0001_1_01_000000_0
2016-01-14 17:16:24,564 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791784281_0001_00_000001: 12 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:24,563 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791784281_0001_1_01_000000_0
2016-01-14 17:16:24,565 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:24,563 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791784281_0001_1_01_000000_0] started. Is using containerId: [container_1452791784281_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:24,565 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:24,565 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:24,564 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791784281_0001_00_000001, Exiting
2016-01-14 17:16:24,566 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791784281_0001_00_000001
2016-01-14 17:16:24,566 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791784281_0001_00_000001
2016-01-14 17:16:24,566 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:24,569 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:24,569 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791784281_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:24,569 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:24,569 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791784281_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F88F5CCAA8AA43569D44AF583AC1357D, taskAttemptId=attempt_1452791784281_0001_1_01_000000_0, startTime=1452791784563, containerId=container_1452791784281_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791784281_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:24,570 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791784281_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:24,571 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791784281_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:24,570 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791784281_0001_00_000001 completed successfully
2016-01-14 17:16:24,571 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791784281_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:24,570 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:24,571 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [DBD5D3CB4C6B4869B80820C5DA0FFE69/F5B5C0E4F3364F9187441605FF7F2311] every/(1/1) ...atform/-1_UNKNOWN_RESULTS, VertexName: F88F5CCAA8AA43569D44AF583AC1357D, VertexParallelism: 1, TaskAttemptID:attempt_1452791784281_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A1526F8B76874BB384D1F52C001AB8BA, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=E88468F2E09048D19120F6ABAC84E963, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:24,572 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:16:24,572 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791784281_0001_1_01_000000_0
2016-01-14 17:16:24,573 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:24,573 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:24,573 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791784281_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:24,573 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=A1526F8B76874BB384D1F52C001AB8BA, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:24,574 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=E88468F2E09048D19120F6ABAC84E963, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:24,574 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:24,574 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:24,574 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: A1526F8B76874BB384D1F52C001AB8BA
2016-01-14 17:16:24,575 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:24,575 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: E88468F2E09048D19120F6ABAC84E963
2016-01-14 17:16:24,575 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:24,576 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: E88468F2E09048D19120F6ABAC84E963
2016-01-14 17:16:24,576 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:24,577 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:24,577 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: A1526F8B76874BB384D1F52C001AB8BA
2016-01-14 17:16:24,578 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: A1526F8B76874BB384D1F52C001AB8BA
2016-01-14 17:16:24,578 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:24,588 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F88F5CCAA8AA43569D44AF583AC1357D, ordinal: 1
2016-01-14 17:16:24,588 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F88F5CCAA8AA43569D44AF583AC1357D, mem on start (mb), free: 121, total: 490, max: 672
2016-01-14 17:16:24,588 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:24,589 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:24,589 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:24,589 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: E88468F2E09048D19120F6ABAC84E963
2016-01-14 17:16:24,589 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:24,589 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:24,589 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:24,589 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:24,590 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:24,590 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:24,590 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:24,590 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:24,590 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, E88468F2E09048D19120F6ABAC84E963, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:24,590 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, A1526F8B76874BB384D1F52C001AB8BA, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 17:16:24,590 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:24,590 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: A1526F8B76874BB384D1F52C001AB8BA
2016-01-14 17:16:24,590 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: A1526F8B76874BB384D1F52C001AB8BA being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:24,591 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:24,592 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:24,593 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:24,593 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: A1526F8B76874BB384D1F52C001AB8BA with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:24,594 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 17:16:24,594 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:24,594 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:24,593 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:24,595 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: A1526F8B76874BB384D1F52C001AB8BA
2016-01-14 17:16:24,595 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:24,597 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 17:16:24,597 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791784281_0001_1_01_000000_0
2016-01-14 17:16:24,597 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 17:16:24,599 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791784281_0001_1_00_000000_0_10039, runDuration: 47059, ]
2016-01-14 17:16:24,600 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791784281_0001_1_00_000000_0_10039] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791784600, TimeTaken=0, Rate=0.00 MB/s
2016-01-14 17:16:24,600 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : A1526F8B76874BB384D1F52C001AB8BA
2016-01-14 17:16:24,601 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:24,601 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:24,601 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [A1526F8B76874BB384D1F52C001AB8BA] #7 in 1ms
2016-01-14 17:16:24,601 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [A1526F8B76874BB384D1F52C001AB8BA] #7, status:false, isInterrupted:false
2016-01-14 17:16:24,601 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: AC463448BFE54A429469C97217F90E84
2016-01-14 17:16:24,602 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:24,602 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_RESULTS"], id: E88468F2E09048D19120F6ABAC84E963
2016-01-14 17:16:24,602 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:24,602 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:24,602 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:24,602 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:24,603 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : A1526F8B76874BB384D1F52C001AB8BA
2016-01-14 17:16:24,602 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_RESULTS"]
2016-01-14 17:16:24,603 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:24,603 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] AC463448BFE54A429469C97217F90E84, for 1 inputs
2016-01-14 17:16:24,604 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F88F5CCAA8AA43569D44AF583AC1357D, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:24,604 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@3dbd95d1; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@69b6ec01
2016-01-14 17:16:24,610 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:24,611 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:24,611 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791784281_0001_1_01_000000_0
2016-01-14 17:16:24,612 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791784281_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:24,612 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791784281_0001_1_01_000000_0
2016-01-14 17:16:24,613 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F88F5CCAA8AA43569D44AF583AC1357D, mem on close (mb), free: 117, total: 490, max: 672
2016-01-14 17:16:24,613 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: A1526F8B76874BB384D1F52C001AB8BA
2016-01-14 17:16:24,613 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:24,613 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:24,613 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:24,613 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:24,614 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:24 Completed running task attempt: attempt_1452791784281_0001_1_01_000000_0
2016-01-14 17:16:24,615 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791784281_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:24,616 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=652, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791784604, Process_Duration=8, Process_End_Time=1452791784612, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 17:16:24,616 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:24,616 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:24,616 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791784281_0001_00_000002
2016-01-14 17:16:24,616 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791784281_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F88F5CCAA8AA43569D44AF583AC1357D, taskAttemptId=attempt_1452791784281_0001_1_01_000000_0, startTime=1452791784563, finishTime=1452791784616, timeTaken=53, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=652, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791784604, Process_Duration=8, Process_End_Time=1452791784612, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:24,617 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791784281_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:24,617 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:24,617 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791784281_0001_1_01_000000_0
2016-01-14 17:16:24,618 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791784281_0001_1][Event:TASK_FINISHED]: vertexName=F88F5CCAA8AA43569D44AF583AC1357D, taskId=task_1452791784281_0001_1_01_000000, startTime=1452791784563, finishTime=1452791784617, timeTaken=54, status=SUCCEEDED, successfulAttemptID=attempt_1452791784281_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=652, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791784604, Process_Duration=8, Process_End_Time=1452791784612, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:24,616 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:24,618 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791784281_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:24,618 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791784281_0001_1_01 [F88F5CCAA8AA43569D44AF583AC1357D] : 1
2016-01-14 17:16:24,618 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791784281_0001_1_01 [F88F5CCAA8AA43569D44AF583AC1357D]
2016-01-14 17:16:24,619 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:24,620 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:24,620 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791784281_0001_1][Event:VERTEX_FINISHED]: vertexName=F88F5CCAA8AA43569D44AF583AC1357D, vertexId=vertex_1452791784281_0001_1_01, initRequestedTime=1452791784458, initedTime=1452791784467, startRequestedTime=1452791784475, startedTime=1452791784475, finishTime=1452791784618, timeTaken=143, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=652, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791784604, Process_Duration=8, Process_End_Time=1452791784612, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452791784563, firstTasksToStart=[ task_1452791784281_0001_1_01_000000 ], lastTaskFinishTime=1452791784616, lastTasksToFinish=[ task_1452791784281_0001_1_01_000000 ], minTaskDuration=53, maxTaskDuration=53, avgTaskDuration=53.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791784281_0001_1_01_000000 ], longestDurationTasks=[ task_1452791784281_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:24,620 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791784281_0001_1_01 [F88F5CCAA8AA43569D44AF583AC1357D] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:24,620 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791784281_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:24,621 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791784281_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791784281_0001_00_000002, stoppedTime=1452791784621, exitStatus=0
2016-01-14 17:16:24,621 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:24,621 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791784281_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:24,622 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791784281_0001_1_01 [F88F5CCAA8AA43569D44AF583AC1357D] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:24,622 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791784281_0001_1
2016-01-14 17:16:24,622 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791784281_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452791784281_0001_1
2016-01-14 17:16:24,622 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: E88468F2E09048D19120F6ABAC84E963 for vertex: vertex_1452791784281_0001_1_01 [F88F5CCAA8AA43569D44AF583AC1357D]
2016-01-14 17:16:24,626 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791784281_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 17:16:24,626 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791784281_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:24,626 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791784281_0001_00_000002, Exiting
2016-01-14 17:16:24,627 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791784281_0001_00_000002
2016-01-14 17:16:24,627 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791784281_0001_00_000002
2016-01-14 17:16:24,627 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791784281_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:24,627 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791784281_0001_00_000002 completed successfully
2016-01-14 17:16:24,630 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452791784281_0001_1_00 [A1526F8B76874BB384D1F52C001AB8BA]
2016-01-14 17:16:24,632 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791784281_0001_1, queueSize=0
2016-01-14 17:16:24,632 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:24,633 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791784281_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791784281_0001_1, startTime=1452791784455, finishTime=1452791784630, timeTaken=175, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=985, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1028653056, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905583569141, Process_Duration=10, Process_End_Time=2905583569151, Read_Duration=2, Tuples_Read=40, Tuples_Written=40, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 17:16:24,633 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791784281_0001_1 finished with state: SUCCEEDED
2016-01-14 17:16:24,633 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791784281_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:24,633 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791784281_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:16:24,633 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452791784281_0001_00_000002
2016-01-14 17:16:24 Completed Dag: dag_1452791784281_0001_1
2016-01-14 17:16:24,635 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791784281_0001_1, dagState=SUCCEEDED
2016-01-14 17:16:24,636 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791784281_0001_1_post
2016-01-14 17:16:24,636 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:24,636 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:24,641 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...atform/-1_UNKNOWN_RESULTS, applicationId=application_1452791784281_0001
2016-01-14 17:16:24,644 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:24,644 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:24,644 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:24,644 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:24,644 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:24,645 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:24,645 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:24,645 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:24,645 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:24,645 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:24,646 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:24,646 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:24,646 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:24,650 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/DBD5D3CB4C6B4869B80820C5DA0FFE69/F5B5C0E4F3364F9187441605FF7F2311/.tez/application_1452791784281_0001
2016-01-14 17:16:24,651 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.391
2016-01-14 17:16:24,721 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_UNKNOWN_RESULTS
2016-01-14 17:16:24,744 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:24,744 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:24,783 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-01-14 17:16:24,791 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.022
2016-01-14 17:16:24,791 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:24,792 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:24,792 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:24,806 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:24,813 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_ALL
2016-01-14 17:16:24,872 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:24,873 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:24,874 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_ALL"]
2016-01-14 17:16:24,874 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:24,874 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:24,875 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:24,877 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...lyplatform/-1_UNKNOWN_ALL
2016-01-14 17:16:24,877 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:24,892 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:24,897 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:24,898 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:24,898 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:24,908 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/C7D87E9060114DD1AE920EAE6E827946/F3F685FB970C4514B8DE72CFFD608F02/.tez/application_1452791784898_0001 doesn't exist and is created
2016-01-14 17:16:24,926 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:24,926 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:24,927 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/C7D87E9060114DD1AE920EAE6E827946/F3F685FB970C4514B8DE72CFFD608F02/.tez/application_1452791784898_0001
2016-01-14 17:16:24,931 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791784898_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:24,942 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:24,943 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:24,944 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:24,945 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:24,945 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/C7D87E9060114DD1AE920EAE6E827946/F3F685FB970C4514B8DE72CFFD608F02/.tez/application_1452791784898_0001/localmode-log-dir
2016-01-14 17:16:24,945 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/C7D87E9060114DD1AE920EAE6E827946/F3F685FB970C4514B8DE72CFFD608F02/.tez/application_1452791784898_0001/localmode-log-dir/history.txt.appattempt_1452791784898_0001_000000, maxErrors=10
2016-01-14 17:16:24,945 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:24,946 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791784898_0001_000000, appSubmitTime=1452791784931, launchTime=1452791784931
2016-01-14 17:16:24,950 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:24,951 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:24,952 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:24,952 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:44001
2016-01-14 17:16:24,954 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:24,955 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:24,958 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791784898_0001_000000, startTime=1452791784958
2016-01-14 17:16:24,958 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:24,958 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:25,026 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:25,027 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:25,027 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...lyplatform/-1_UNKNOWN_ALL, applicationId=application_1452791784898_0001, dagName=[C7D87E9060114DD1AE920EAE6E827946/F3F685FB970C4514B8DE72CFFD608F02] every/(1/1) ...lyplatform/-1_UNKNOWN_ALL
2016-01-14 17:16:25,054 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [C7D87E9060114DD1AE920EAE6E827946/F3F685FB970C4514B8DE72CFFD608F02] every/(1/1) ...lyplatform/-1_UNKNOWN_ALL
2016-01-14 17:16:25,056 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791784898_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/C7D87E9060114DD1AE920EAE6E827946/F3F685FB970C4514B8DE72CFFD608F02/.tez/application_1452791784898_0001/localmode-log-dir/dag_1452791784898_0001_1.dot
2016-01-14 17:16:25,057 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/C7D87E9060114DD1AE920EAE6E827946/F3F685FB970C4514B8DE72CFFD608F02/.tez/application_1452791784898_0001/localmode-log-dir/dag_1452791784898_0001_1-tez-dag.pb.txt
2016-01-14 17:16:25,074 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791784898_0001_1
2016-01-14 17:16:25,074 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:25,074 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [C7D87E9060114DD1AE920EAE6E827946/F3F685FB970C4514B8DE72CFFD608F02] every/(1/1) ...lyplatform/-1_UNKNOWN_ALL
2016-01-14 17:16:25 Running Dag: dag_1452791784898_0001_1
2016-01-14 17:16:25,089 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:25,090 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791784898_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791784898_0001_1, submitTime=1452791785055
2016-01-14 17:16:25,091 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex DA065AAE08144533BE5395EA687BB247
2016-01-14 17:16:25,091 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:25,092 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:25,093 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791784898_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791784898_0001_1, initTime=1452791785090
2016-01-14 17:16:25,093 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791784898_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:25,093 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791784898_0001_1][Event:DAG_STARTED]: dagID=dag_1452791784898_0001_1, startTime=1452791785093
2016-01-14 17:16:25,093 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:25,093 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:25,094 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791784898_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:25,094 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: B167B67765854CEDA0C81F2D2F411366 : {EBB304FD878B48CEBA9530E6A281EB7D={InputName=EBB304FD878B48CEBA9530E6A281EB7D}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:25,094 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: EBB304FD878B48CEBA9530E6A281EB7D, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:25,094 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791784898_0001_1_00 [B167B67765854CEDA0C81F2D2F411366]
2016-01-14 17:16:25,094 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791784898_0001_1_00 [B167B67765854CEDA0C81F2D2F411366]
2016-01-14 17:16:25,095 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791784898_0001_1_00 [B167B67765854CEDA0C81F2D2F411366]
2016-01-14 17:16:25,095 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791784898_0001_1_00 [B167B67765854CEDA0C81F2D2F411366]. Starting root input initializers: 1
2016-01-14 17:16:25,095 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...lyplatform/-1_UNKNOWN_ALL, applicationId=application_1452791784898_0001, dagName=[C7D87E9060114DD1AE920EAE6E827946/F3F685FB970C4514B8DE72CFFD608F02] every/(1/1) ...lyplatform/-1_UNKNOWN_ALL
2016-01-14 17:16:25,095 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791784898_0001_1_00 [B167B67765854CEDA0C81F2D2F411366] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:25,095 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791784898_0001_1_01 [DA065AAE08144533BE5395EA687BB247]
2016-01-14 17:16:25,098 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: EBB304FD878B48CEBA9530E6A281EB7D on vertex vertex_1452791784898_0001_1_00 [B167B67765854CEDA0C81F2D2F411366]
2016-01-14 17:16:25,099 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:25,099 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791784898_0001_1_01 [DA065AAE08144533BE5395EA687BB247]
2016-01-14 17:16:25,099 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791784898_0001_1_01 [DA065AAE08144533BE5395EA687BB247]
2016-01-14 17:16:25,099 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791784898_0001_1_01 [DA065AAE08144533BE5395EA687BB247]
2016-01-14 17:16:25,099 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=6A87A5782FBD4862BF61C89D667BAF95, vertexId=vertex_1452791784898_0001_1_01 [DA065AAE08144533BE5395EA687BB247], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:25,100 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=6A87A5782FBD4862BF61C89D667BAF95, vertexId=vertex_1452791784898_0001_1_01 [DA065AAE08144533BE5395EA687BB247]
2016-01-14 17:16:25,100 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:25,100 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input EBB304FD878B48CEBA9530E6A281EB7D asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:25,102 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:25,102 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:25,103 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=6A87A5782FBD4862BF61C89D667BAF95, vertexName=DA065AAE08144533BE5395EA687BB247, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:25,103 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=6A87A5782FBD4862BF61C89D667BAF95, vertexId=vertex_1452791784898_0001_1_01 [DA065AAE08144533BE5395EA687BB247]
2016-01-14 17:16:25,106 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:25,106 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791784898_0001_1][Event:VERTEX_INITIALIZED]: vertexName=DA065AAE08144533BE5395EA687BB247, vertexId=vertex_1452791784898_0001_1_01, initRequestedTime=1452791785095, initedTime=1452791785105, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:25,106 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791784898_0001_1_01 [DA065AAE08144533BE5395EA687BB247] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:25,106 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:25,107 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: EBB304FD878B48CEBA9530E6A281EB7D on vertex vertex_1452791784898_0001_1_00 [B167B67765854CEDA0C81F2D2F411366]
2016-01-14 17:16:25,107 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791784898_0001_1_00 [B167B67765854CEDA0C81F2D2F411366] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:25,107 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {EBB304FD878B48CEBA9530E6A281EB7D=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:25,107 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791784898_0001_1_00 [B167B67765854CEDA0C81F2D2F411366] parallelism set to 1
2016-01-14 17:16:25,108 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791784898_0001_1_00 [B167B67765854CEDA0C81F2D2F411366] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:25,108 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791784898_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791784898_0001_1_00, eventCount=1
2016-01-14 17:16:25,109 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:25,109 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791784898_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B167B67765854CEDA0C81F2D2F411366, vertexId=vertex_1452791784898_0001_1_00, initRequestedTime=1452791785094, initedTime=1452791785108, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:25,109 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791784898_0001_1_00 [B167B67765854CEDA0C81F2D2F411366] with distanceFromRoot: 0
2016-01-14 17:16:25,109 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791784898_0001_1_00 [B167B67765854CEDA0C81F2D2F411366] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:25,109 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in B167B67765854CEDA0C81F2D2F411366
2016-01-14 17:16:25,110 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791784898_0001_1_00 [B167B67765854CEDA0C81F2D2F411366]
2016-01-14 17:16:25,110 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791784898_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791784898_0001_1_00, startRequestedTime=1452791785109, startedTime=1452791785109
2016-01-14 17:16:25,110 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:25,111 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791784898_0001_1_00 [B167B67765854CEDA0C81F2D2F411366] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:25,111 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:25,111 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791784898_0001_1][Event:TASK_STARTED]: vertexName=B167B67765854CEDA0C81F2D2F411366, taskId=task_1452791784898_0001_1_00_000000, scheduledTime=1452791785111, launchTime=1452791785111
2016-01-14 17:16:25,112 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791784898_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:25,112 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791784898_0001_1_00 for vertex: vertex_1452791784898_0001_1_01 [DA065AAE08144533BE5395EA687BB247] numStartedSources: 1 numSources: 1
2016-01-14 17:16:25,112 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791784898_0001_1_01 [DA065AAE08144533BE5395EA687BB247] with distanceFromRoot: 1
2016-01-14 17:16:25,113 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791784898_0001_1_00 [B167B67765854CEDA0C81F2D2F411366]
2016-01-14 17:16:25,113 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: B167B67765854CEDA0C81F2D2F411366 in vertex: DA065AAE08144533BE5395EA687BB247
2016-01-14 17:16:25,113 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: DA065AAE08144533BE5395EA687BB247 with 1 source tasks and 1 pending tasks
2016-01-14 17:16:25,114 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791784898_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791784898_0001_1_01, startRequestedTime=1452791785113, startedTime=1452791785113
2016-01-14 17:16:25,114 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:25,114 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791784898_0001_1_01 [DA065AAE08144533BE5395EA687BB247] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:25,115 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [C7D87E9060114DD1AE920EAE6E827946/F3F685FB970C4514B8DE72CFFD608F02] every/(1/1) ...lyplatform/-1_UNKNOWN_ALL, VertexName: B167B67765854CEDA0C81F2D2F411366, VertexParallelism: 1, TaskAttemptID:attempt_1452791784898_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EBB304FD878B48CEBA9530E6A281EB7D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=DA065AAE08144533BE5395EA687BB247, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:25,115 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791784898_0001, with dag id: dag_1452791784898_0001_1
2016-01-14 17:16:25,116 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791784898_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:25,116 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:25,117 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:25,117 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:25,117 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:25,118 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791784898_0001_00_000001
2016-01-14 17:16:25,118 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791784898_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:25,120 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791784898_0001_00_000001
2016-01-14 17:16:25,120 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:25,121 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:25,121 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791784898_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791784898_0001_00_000001, launchTime=1452791785120
2016-01-14 17:16:25,121 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791784898_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:25,122 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791784898_0001_00_000001
2016-01-14 17:16:25,123 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791784898_0001_1_00_000000_0] to container: [container_1452791784898_0001_00_000001]
2016-01-14 17:16:25,124 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791784898_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:25,124 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791784898_0001_00_000001 given task: attempt_1452791784898_0001_1_00_000000_0
2016-01-14 17:16:25,124 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791784898_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791784898_0001_1_00_000000_0
2016-01-14 17:16:25,125 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791784898_0001_1_00_000000_0] started. Is using containerId: [container_1452791784898_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:25,125 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:25,125 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791784898_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B167B67765854CEDA0C81F2D2F411366, taskAttemptId=attempt_1452791784898_0001_1_00_000000_0, startTime=1452791785124, containerId=container_1452791784898_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791784898_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:25,126 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791784898_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:25 Starting to run new task attempt: attempt_1452791784898_0001_1_00_000000_0
2016-01-14 17:16:25,126 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791784898_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:25,126 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791784898_0001_1_00_000000_0
2016-01-14 17:16:25,127 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:25,127 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:25,127 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:25,127 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:25,127 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:25,128 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:25,128 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [C7D87E9060114DD1AE920EAE6E827946/F3F685FB970C4514B8DE72CFFD608F02] every/(1/1) ...lyplatform/-1_UNKNOWN_ALL, VertexName: B167B67765854CEDA0C81F2D2F411366, VertexParallelism: 1, TaskAttemptID:attempt_1452791784898_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EBB304FD878B48CEBA9530E6A281EB7D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=DA065AAE08144533BE5395EA687BB247, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:25,128 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:16:25,129 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791784898_0001_1_00_000000_0
2016-01-14 17:16:25,129 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:25,130 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:25,130 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791784898_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:25,131 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=EBB304FD878B48CEBA9530E6A281EB7D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:25,131 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=DA065AAE08144533BE5395EA687BB247, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:25,131 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:25,132 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:25,131 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:25,132 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: EBB304FD878B48CEBA9530E6A281EB7D
2016-01-14 17:16:25,132 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: EBB304FD878B48CEBA9530E6A281EB7D
2016-01-14 17:16:25,132 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: DA065AAE08144533BE5395EA687BB247
2016-01-14 17:16:25,135 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:25,135 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:25,136 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: EBB304FD878B48CEBA9530E6A281EB7D
2016-01-14 17:16:25,136 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: DA065AAE08144533BE5395EA687BB247
2016-01-14 17:16:25,136 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: EBB304FD878B48CEBA9530E6A281EB7D
2016-01-14 17:16:25,137 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:25,137 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: DA065AAE08144533BE5395EA687BB247
2016-01-14 17:16:25,144 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B167B67765854CEDA0C81F2D2F411366, ordinal: 0
2016-01-14 17:16:25,144 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B167B67765854CEDA0C81F2D2F411366, mem on start (mb), free: 238, total: 491, max: 672
2016-01-14 17:16:25,144 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:25,144 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:25,145 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:25,145 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:25,145 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:25,146 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:25,146 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:25,148 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:25,148 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:25,149 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 17:16:25,149 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, EBB304FD878B48CEBA9530E6A281EB7D, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:25,149 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, DA065AAE08144533BE5395EA687BB247, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:25,150 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:25,150 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: EBB304FD878B48CEBA9530E6A281EB7D
2016-01-14 17:16:25,151 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:25,151 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: EBB304FD878B48CEBA9530E6A281EB7D
2016-01-14 17:16:25,152 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: EBB304FD878B48CEBA9530E6A281EB7D being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:25,152 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:25,153 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:25,153 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:25,154 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791784898_0001_1_00_000000_0
2016-01-14 17:16:25,154 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:25,155 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@7346596e
2016-01-14 17:16:25,156 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@7346596e
2016-01-14 17:16:25,157 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:25,157 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:25,158 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:25,159 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: EBB304FD878B48CEBA9530E6A281EB7D
2016-01-14 17:16:25,159 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 3BDB76FC4B5645379FE1945A7096AD65
2016-01-14 17:16:25,160 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 3BDB76FC4B5645379FE1945A7096AD65
2016-01-14 17:16:25,160 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@3b8cdffe; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@1dc5bb3d; comparator=cascading.tuple.hadoop.util.TupleComparator@2087b110; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:25,161 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:25,161 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:25,161 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:25,175 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:25,175 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:25,176 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:25,176 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:25,176 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:25,177 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:25,177 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:25,179 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B167B67765854CEDA0C81F2D2F411366, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:25,182 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791784898_0001_1_00_000000_0
2016-01-14 17:16:25,182 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B167B67765854CEDA0C81F2D2F411366, mem on close (mb), free: 136, total: 491, max: 672
2016-01-14 17:16:25,183 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:25,183 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:25,183 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:25,183 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:25,186 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:25 Completed running task attempt: attempt_1452791784898_0001_1_00_000000_0
2016-01-14 17:16:25,191 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791784898_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:25,192 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791784898_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791784898_0001_1_00, eventCount=1
2016-01-14 17:16:25,193 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:25,193 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791784898_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B167B67765854CEDA0C81F2D2F411366, taskAttemptId=attempt_1452791784898_0001_1_00_000000_0, startTime=1452791785124, finishTime=1452791785192, timeTaken=68, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514850816, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791785179, Process_Duration=3, Process_End_Time=1452791785182, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:25,193 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791784898_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:25,193 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791784898_0001_1_00_000000_0
2016-01-14 17:16:25,194 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791784898_0001_1][Event:TASK_FINISHED]: vertexName=B167B67765854CEDA0C81F2D2F411366, taskId=task_1452791784898_0001_1_00_000000, startTime=1452791785124, finishTime=1452791785193, timeTaken=69, status=SUCCEEDED, successfulAttemptID=attempt_1452791784898_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514850816, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791785179, Process_Duration=3, Process_End_Time=1452791785182, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:25,194 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:25,194 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791784898_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:25,194 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791784898_0001_1_00 [B167B67765854CEDA0C81F2D2F411366] : 1
2016-01-14 17:16:25,195 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791784898_0001_1_00 [B167B67765854CEDA0C81F2D2F411366]
2016-01-14 17:16:25,195 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:25,197 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:25,197 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791784898_0001_1][Event:VERTEX_FINISHED]: vertexName=B167B67765854CEDA0C81F2D2F411366, vertexId=vertex_1452791784898_0001_1_00, initRequestedTime=1452791785094, initedTime=1452791785108, startRequestedTime=1452791785109, startedTime=1452791785109, finishTime=1452791785195, timeTaken=86, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514850816, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791785179, Process_Duration=3, Process_End_Time=1452791785182, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791785124, firstTasksToStart=[ task_1452791784898_0001_1_00_000000 ], lastTaskFinishTime=1452791785192, lastTasksToFinish=[ task_1452791784898_0001_1_00_000000 ], minTaskDuration=68, maxTaskDuration=68, avgTaskDuration=68.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791784898_0001_1_00_000000 ], longestDurationTasks=[ task_1452791784898_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:25,197 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791784898_0001_1_00 [B167B67765854CEDA0C81F2D2F411366] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:25,198 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514850816, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791785179, Process_Duration=3, Process_End_Time=1452791785182, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:25,198 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791784898_0001_1_01 [DA065AAE08144533BE5395EA687BB247] attempt: attempt_1452791784898_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:25,198 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: DA065AAE08144533BE5395EA687BB247
2016-01-14 17:16:25,199 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791784898_0001_1_01 [DA065AAE08144533BE5395EA687BB247]
2016-01-14 17:16:25,198 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:25,199 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:25,199 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791784898_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:25,200 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791784898_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791784898_0001_00_000001, stoppedTime=1452791785199, exitStatus=0
2016-01-14 17:16:25,200 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:25,200 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791784898_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:25,200 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791784898_0001_1_00 [B167B67765854CEDA0C81F2D2F411366] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:25,200 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791784898_0001_00_000001
2016-01-14 17:16:25,200 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791784898_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:25,201 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791784898_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:25,202 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791784898_0001_00_000001, Exiting
2016-01-14 17:16:25,201 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791784898_0001_1][Event:TASK_STARTED]: vertexName=DA065AAE08144533BE5395EA687BB247, taskId=task_1452791784898_0001_1_01_000000, scheduledTime=1452791785200, launchTime=1452791785200
2016-01-14 17:16:25,201 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:25,202 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791784898_0001_00_000001
2016-01-14 17:16:25,203 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791784898_0001_00_000001
2016-01-14 17:16:25,203 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791784898_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:25,205 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791784898_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:25,206 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791784898_0001_1_01 [DA065AAE08144533BE5395EA687BB247]
2016-01-14 17:16:25,206 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791784898_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:25,206 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [C7D87E9060114DD1AE920EAE6E827946/F3F685FB970C4514B8DE72CFFD608F02] every/(1/1) ...lyplatform/-1_UNKNOWN_ALL, VertexName: DA065AAE08144533BE5395EA687BB247, VertexParallelism: 1, TaskAttemptID:attempt_1452791784898_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B167B67765854CEDA0C81F2D2F411366, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=6A87A5782FBD4862BF61C89D667BAF95, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:25,206 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791784898_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:25,207 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:25,209 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791784898_0001_00_000002
2016-01-14 17:16:25,209 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791784898_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:25,209 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791784898_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:25,210 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791784898_0001_00_000001 completed successfully
2016-01-14 17:16:25,211 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791784898_0001_00_000002
2016-01-14 17:16:25,212 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791784898_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:25,212 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 17:16:25,218 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791784898_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:25,218 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:25,219 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791784898_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791784898_0001_00_000002, launchTime=1452791785218
2016-01-14 17:16:25,220 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791784898_0001_00_000002
2016-01-14 17:16:25,221 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791784898_0001_1_01_000000_0] to container: [container_1452791784898_0001_00_000002]
2016-01-14 17:16:25,221 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791784898_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:25,221 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791784898_0001_1_01_000000_0] started. Is using containerId: [container_1452791784898_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:25,221 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791784898_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=DA065AAE08144533BE5395EA687BB247, taskAttemptId=attempt_1452791784898_0001_1_01_000000_0, startTime=1452791785221, containerId=container_1452791784898_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791784898_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:25,221 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791784898_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:25,222 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791784898_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:25,222 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:25,222 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791784898_0001_00_000002 given task: attempt_1452791784898_0001_1_01_000000_0
2016-01-14 17:16:25,222 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791784898_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791784898_0001_1_01_000000_0
2016-01-14 17:16:25 Starting to run new task attempt: attempt_1452791784898_0001_1_01_000000_0
2016-01-14 17:16:25,223 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791784898_0001_1_01_000000_0
2016-01-14 17:16:25,224 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:25,224 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:25,224 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:25,224 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:25,224 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:25,225 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:25,225 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [C7D87E9060114DD1AE920EAE6E827946/F3F685FB970C4514B8DE72CFFD608F02] every/(1/1) ...lyplatform/-1_UNKNOWN_ALL, VertexName: DA065AAE08144533BE5395EA687BB247, VertexParallelism: 1, TaskAttemptID:attempt_1452791784898_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B167B67765854CEDA0C81F2D2F411366, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=6A87A5782FBD4862BF61C89D667BAF95, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:25,225 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:16:25,226 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791784898_0001_1_01_000000_0
2016-01-14 17:16:25,226 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:25,226 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:25,227 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:25,229 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:25,232 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=6A87A5782FBD4862BF61C89D667BAF95, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:25,233 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B167B67765854CEDA0C81F2D2F411366, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:25,234 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:25,235 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B167B67765854CEDA0C81F2D2F411366
2016-01-14 17:16:25,234 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:25,236 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 6A87A5782FBD4862BF61C89D667BAF95
2016-01-14 17:16:25,238 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:25,243 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 6A87A5782FBD4862BF61C89D667BAF95
2016-01-14 17:16:25,237 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:25,245 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:25,245 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: B167B67765854CEDA0C81F2D2F411366
2016-01-14 17:16:25,238 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: DA065AAE08144533BE5395EA687BB247, ordinal: 1
2016-01-14 17:16:25,248 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B167B67765854CEDA0C81F2D2F411366
2016-01-14 17:16:25,246 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791784898_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:25,248 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DA065AAE08144533BE5395EA687BB247, mem on start (mb), free: 129, total: 491, max: 672
2016-01-14 17:16:25,249 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:25,249 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:25,249 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:25,257 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:25,257 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 6A87A5782FBD4862BF61C89D667BAF95
2016-01-14 17:16:25,257 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:25,257 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:25,257 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:25,258 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:25,258 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:25,258 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:25,258 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:25,258 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 6A87A5782FBD4862BF61C89D667BAF95, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:25,258 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B167B67765854CEDA0C81F2D2F411366, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 17:16:25,258 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:25,258 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B167B67765854CEDA0C81F2D2F411366 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:25,258 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:25,258 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B167B67765854CEDA0C81F2D2F411366
2016-01-14 17:16:25,258 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:25,259 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:25,260 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:25,260 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: B167B67765854CEDA0C81F2D2F411366 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:25,261 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 17:16:25,261 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:25,261 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:25,262 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B167B67765854CEDA0C81F2D2F411366
2016-01-14 17:16:25,262 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:25,263 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791784898_0001_1_01_000000_0
2016-01-14 17:16:25,263 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791784898_0001_1_00_000000_0_10045, runDuration: 55497, ]
2016-01-14 17:16:25,265 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791784898_0001_1_00_000000_0_10045] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791785265, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 17:16:25,266 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:25,266 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : B167B67765854CEDA0C81F2D2F411366
2016-01-14 17:16:25,266 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:25,266 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [B167B67765854CEDA0C81F2D2F411366] #8 in 2ms
2016-01-14 17:16:25,266 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [B167B67765854CEDA0C81F2D2F411366] #8, status:false, isInterrupted:false
2016-01-14 17:16:25,267 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 3BDB76FC4B5645379FE1945A7096AD65
2016-01-14 17:16:25,268 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:25,269 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_ALL"], id: 6A87A5782FBD4862BF61C89D667BAF95
2016-01-14 17:16:25,269 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:25,269 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:25,269 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_ALL"]
2016-01-14 17:16:25,270 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 3BDB76FC4B5645379FE1945A7096AD65, for 1 inputs
2016-01-14 17:16:25,269 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:25,271 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:25,271 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : B167B67765854CEDA0C81F2D2F411366
2016-01-14 17:16:25,271 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:25,272 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: DA065AAE08144533BE5395EA687BB247, all 1 inputs ready in: 00:00:00.001
2016-01-14 17:16:25,272 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@3dffa16e; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@59bf5c8e
2016-01-14 17:16:25,276 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:25,276 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:25,277 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791784898_0001_1_01_000000_0
2016-01-14 17:16:25,277 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791784898_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:25,278 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791784898_0001_1_01_000000_0
2016-01-14 17:16:25,278 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DA065AAE08144533BE5395EA687BB247, mem on close (mb), free: 126, total: 491, max: 672
2016-01-14 17:16:25,278 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: B167B67765854CEDA0C81F2D2F411366
2016-01-14 17:16:25,278 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:25,279 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:25,280 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:25,280 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:25,280 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:25 Completed running task attempt: attempt_1452791784898_0001_1_01_000000_0
2016-01-14 17:16:25,281 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791784898_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:25,282 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514850816, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791785272, Process_Duration=6, Process_End_Time=1452791785278, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 17:16:25,282 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791784898_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=DA065AAE08144533BE5395EA687BB247, taskAttemptId=attempt_1452791784898_0001_1_01_000000_0, startTime=1452791785221, finishTime=1452791785282, timeTaken=61, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514850816, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791785272, Process_Duration=6, Process_End_Time=1452791785278, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:25,282 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:25,283 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:25,283 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791784898_0001_00_000002
2016-01-14 17:16:25,283 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:25,283 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791784898_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:25,284 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791784898_0001_1_01_000000_0
2016-01-14 17:16:25,285 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791784898_0001_1][Event:TASK_FINISHED]: vertexName=DA065AAE08144533BE5395EA687BB247, taskId=task_1452791784898_0001_1_01_000000, startTime=1452791785221, finishTime=1452791785284, timeTaken=63, status=SUCCEEDED, successfulAttemptID=attempt_1452791784898_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514850816, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791785272, Process_Duration=6, Process_End_Time=1452791785278, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:25,285 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791784898_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:25,285 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791784898_0001_1_01 [DA065AAE08144533BE5395EA687BB247] : 1
2016-01-14 17:16:25,285 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791784898_0001_1_01 [DA065AAE08144533BE5395EA687BB247]
2016-01-14 17:16:25,286 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791784898_0001_1][Event:VERTEX_FINISHED]: vertexName=DA065AAE08144533BE5395EA687BB247, vertexId=vertex_1452791784898_0001_1_01, initRequestedTime=1452791785095, initedTime=1452791785105, startRequestedTime=1452791785113, startedTime=1452791785113, finishTime=1452791785285, timeTaken=172, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514850816, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791785272, Process_Duration=6, Process_End_Time=1452791785278, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452791785221, firstTasksToStart=[ task_1452791784898_0001_1_01_000000 ], lastTaskFinishTime=1452791785282, lastTasksToFinish=[ task_1452791784898_0001_1_01_000000 ], minTaskDuration=61, maxTaskDuration=61, avgTaskDuration=61.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791784898_0001_1_01_000000 ], longestDurationTasks=[ task_1452791784898_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:25,287 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791784898_0001_1_01 [DA065AAE08144533BE5395EA687BB247] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:25,287 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791784898_0001_1_01 [DA065AAE08144533BE5395EA687BB247] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:25,287 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791784898_0001_1
2016-01-14 17:16:25,287 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791784898_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452791784898_0001_1
2016-01-14 17:16:25,287 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 6A87A5782FBD4862BF61C89D667BAF95 for vertex: vertex_1452791784898_0001_1_01 [DA065AAE08144533BE5395EA687BB247]
2016-01-14 17:16:25,288 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:25,288 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:25,288 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:25,298 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452791784898_0001_1_00 [B167B67765854CEDA0C81F2D2F411366]
2016-01-14 17:16:25,300 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791784898_0001_1, queueSize=0
2016-01-14 17:16:25,301 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:25,301 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791784898_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791784898_0001_1, startTime=1452791785093, finishTime=1452791785298, timeTaken=205, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=571, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1029701632, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905583570451, Process_Duration=9, Process_End_Time=2905583570460, Read_Duration=3, Tuples_Read=40, Tuples_Written=40, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 17:16:25,302 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791784898_0001_1 finished with state: SUCCEEDED
2016-01-14 17:16:25,302 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791784898_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:25,302 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791784898_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:25,302 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791784898_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791784898_0001_00_000002, stoppedTime=1452791785302, exitStatus=0
2016-01-14 17:16:25,302 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:25,303 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791784898_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:25 Completed Dag: dag_1452791784898_0001_1
2016-01-14 17:16:25,303 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791784898_0001_1, dagState=SUCCEEDED
2016-01-14 17:16:25,303 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791784898_0001_1_post
2016-01-14 17:16:25,303 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:25,304 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:25,304 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791784898_0001_00_000002. Relying on regular task shutdown for it to end
2016-01-14 17:16:25,304 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791784898_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:25,311 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...lyplatform/-1_UNKNOWN_ALL, applicationId=application_1452791784898_0001
2016-01-14 17:16:25,313 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:25,313 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:25,314 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:25,314 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:25,314 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:25,314 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791784898_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 17:16:25,314 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791784898_0001_00_000002: 31 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:25,314 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791784898_0001_00_000002, Exiting
2016-01-14 17:16:25,315 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791784898_0001_00_000002
2016-01-14 17:16:25,315 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791784898_0001_00_000002
2016-01-14 17:16:25,315 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:25,315 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791784898_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:25,316 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791784898_0001_00_000002 completed successfully
2016-01-14 17:16:25,316 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:25,316 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791784898_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:25,317 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:25,317 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:25,317 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:25,317 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:25,318 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:25,318 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:25,324 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/C7D87E9060114DD1AE920EAE6E827946/F3F685FB970C4514B8DE72CFFD608F02/.tez/application_1452791784898_0001
2016-01-14 17:16:25,324 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.452
2016-01-14 17:16:25,396 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_UNKNOWN_ALL
2016-01-14 17:16:25,419 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:25,420 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:25,460 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.017
2016-01-14 17:16:25,474 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.029
2016-01-14 17:16:25,474 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:25,475 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:25,475 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:25,493 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:25,502 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_REPLACE
2016-01-14 17:16:25,546 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:25,546 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:25,547 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_REPLACE"]
2016-01-14 17:16:25,547 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:25,547 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:25,547 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:25,548 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...atform/-1_UNKNOWN_REPLACE
2016-01-14 17:16:25,549 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:25,560 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:25,565 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:25,566 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:25,566 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:25,578 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/2278A8A555414C9987A3DB580078E1A0/F1016629791B47B196AB1E6754DFB850/.tez/application_1452791785566_0001 doesn't exist and is created
2016-01-14 17:16:25,595 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:25,595 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:25,595 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/2278A8A555414C9987A3DB580078E1A0/F1016629791B47B196AB1E6754DFB850/.tez/application_1452791785566_0001
2016-01-14 17:16:25,598 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791785566_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:25,610 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:25,611 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:25,614 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:25,615 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:25,615 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/2278A8A555414C9987A3DB580078E1A0/F1016629791B47B196AB1E6754DFB850/.tez/application_1452791785566_0001/localmode-log-dir
2016-01-14 17:16:25,616 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/2278A8A555414C9987A3DB580078E1A0/F1016629791B47B196AB1E6754DFB850/.tez/application_1452791785566_0001/localmode-log-dir/history.txt.appattempt_1452791785566_0001_000000, maxErrors=10
2016-01-14 17:16:25,616 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:25,617 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791785566_0001_000000, appSubmitTime=1452791785598, launchTime=1452791785598
2016-01-14 17:16:25,623 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:25,623 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:25,624 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:53492
2016-01-14 17:16:25,626 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:25,626 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:25,627 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:25,629 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791785566_0001_000000, startTime=1452791785629
2016-01-14 17:16:25,629 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:25,629 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:25,686 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 17:16:25,687 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 17:16:25,695 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:25,695 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:25,695 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...atform/-1_UNKNOWN_REPLACE, applicationId=application_1452791785566_0001, dagName=[2278A8A555414C9987A3DB580078E1A0/F1016629791B47B196AB1E6754DFB850] every/(1/1) ...atform/-1_UNKNOWN_REPLACE
2016-01-14 17:16:25,712 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [2278A8A555414C9987A3DB580078E1A0/F1016629791B47B196AB1E6754DFB850] every/(1/1) ...atform/-1_UNKNOWN_REPLACE
2016-01-14 17:16:25,714 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791785566_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/2278A8A555414C9987A3DB580078E1A0/F1016629791B47B196AB1E6754DFB850/.tez/application_1452791785566_0001/localmode-log-dir/dag_1452791785566_0001_1.dot
2016-01-14 17:16:25,714 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/2278A8A555414C9987A3DB580078E1A0/F1016629791B47B196AB1E6754DFB850/.tez/application_1452791785566_0001/localmode-log-dir/dag_1452791785566_0001_1-tez-dag.pb.txt
2016-01-14 17:16:25,721 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791785566_0001_1
2016-01-14 17:16:25,722 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:25,722 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [2278A8A555414C9987A3DB580078E1A0/F1016629791B47B196AB1E6754DFB850] every/(1/1) ...atform/-1_UNKNOWN_REPLACE
2016-01-14 17:16:25 Running Dag: dag_1452791785566_0001_1
2016-01-14 17:16:25,736 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:25,737 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791785566_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791785566_0001_1, submitTime=1452791785713
2016-01-14 17:16:25,738 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex A03ACB1ABF6542DDBDEDB2A899735A26
2016-01-14 17:16:25,739 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:25,740 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:25,740 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791785566_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791785566_0001_1, initTime=1452791785737
2016-01-14 17:16:25,740 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791785566_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:25,741 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...atform/-1_UNKNOWN_REPLACE, applicationId=application_1452791785566_0001, dagName=[2278A8A555414C9987A3DB580078E1A0/F1016629791B47B196AB1E6754DFB850] every/(1/1) ...atform/-1_UNKNOWN_REPLACE
2016-01-14 17:16:25,750 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:25,750 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791785566_0001_1][Event:DAG_STARTED]: dagID=dag_1452791785566_0001_1, startTime=1452791785741
2016-01-14 17:16:25,755 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:25,755 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791785566_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:25,755 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: B82796B6922744D188B575C1EA0E9BDD : {C68FF5DFB480473F9C07824022A7DEAF={InputName=C68FF5DFB480473F9C07824022A7DEAF}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:25,755 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: C68FF5DFB480473F9C07824022A7DEAF, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:25,755 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791785566_0001_1_00 [B82796B6922744D188B575C1EA0E9BDD]
2016-01-14 17:16:25,755 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791785566_0001_1_00 [B82796B6922744D188B575C1EA0E9BDD]
2016-01-14 17:16:25,755 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791785566_0001_1_00 [B82796B6922744D188B575C1EA0E9BDD]
2016-01-14 17:16:25,756 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791785566_0001_1_00 [B82796B6922744D188B575C1EA0E9BDD]. Starting root input initializers: 1
2016-01-14 17:16:25,756 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791785566_0001_1_00 [B82796B6922744D188B575C1EA0E9BDD] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:25,757 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791785566_0001_1_01 [A03ACB1ABF6542DDBDEDB2A899735A26]
2016-01-14 17:16:25,759 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: C68FF5DFB480473F9C07824022A7DEAF on vertex vertex_1452791785566_0001_1_00 [B82796B6922744D188B575C1EA0E9BDD]
2016-01-14 17:16:25,760 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:25,760 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:25,760 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791785566_0001_1_01 [A03ACB1ABF6542DDBDEDB2A899735A26]
2016-01-14 17:16:25,761 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input C68FF5DFB480473F9C07824022A7DEAF asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:25,761 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791785566_0001_1_01 [A03ACB1ABF6542DDBDEDB2A899735A26]
2016-01-14 17:16:25,761 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791785566_0001_1_01 [A03ACB1ABF6542DDBDEDB2A899735A26]
2016-01-14 17:16:25,762 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=8A4B00A77F974BF6839CC3C405CF60A9, vertexId=vertex_1452791785566_0001_1_01 [A03ACB1ABF6542DDBDEDB2A899735A26], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:25,762 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:25,762 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=8A4B00A77F974BF6839CC3C405CF60A9, vertexId=vertex_1452791785566_0001_1_01 [A03ACB1ABF6542DDBDEDB2A899735A26]
2016-01-14 17:16:25,762 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:25,764 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=8A4B00A77F974BF6839CC3C405CF60A9, vertexName=A03ACB1ABF6542DDBDEDB2A899735A26, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:25,764 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=8A4B00A77F974BF6839CC3C405CF60A9, vertexId=vertex_1452791785566_0001_1_01 [A03ACB1ABF6542DDBDEDB2A899735A26]
2016-01-14 17:16:25,764 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:25,764 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: C68FF5DFB480473F9C07824022A7DEAF on vertex vertex_1452791785566_0001_1_00 [B82796B6922744D188B575C1EA0E9BDD]
2016-01-14 17:16:25,766 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:25,766 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791785566_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A03ACB1ABF6542DDBDEDB2A899735A26, vertexId=vertex_1452791785566_0001_1_01, initRequestedTime=1452791785757, initedTime=1452791785765, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:25,766 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791785566_0001_1_01 [A03ACB1ABF6542DDBDEDB2A899735A26] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:25,766 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791785566_0001_1_00 [B82796B6922744D188B575C1EA0E9BDD] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:25,767 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {C68FF5DFB480473F9C07824022A7DEAF=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:25,767 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791785566_0001_1_00 [B82796B6922744D188B575C1EA0E9BDD] parallelism set to 1
2016-01-14 17:16:25,767 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791785566_0001_1_00 [B82796B6922744D188B575C1EA0E9BDD] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:25,767 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791785566_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791785566_0001_1_00, eventCount=1
2016-01-14 17:16:25,768 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:25,768 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791785566_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B82796B6922744D188B575C1EA0E9BDD, vertexId=vertex_1452791785566_0001_1_00, initRequestedTime=1452791785755, initedTime=1452791785768, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:25,768 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791785566_0001_1_00 [B82796B6922744D188B575C1EA0E9BDD] with distanceFromRoot: 0
2016-01-14 17:16:25,769 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791785566_0001_1_00 [B82796B6922744D188B575C1EA0E9BDD] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:25,769 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in B82796B6922744D188B575C1EA0E9BDD
2016-01-14 17:16:25,769 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791785566_0001_1_00 [B82796B6922744D188B575C1EA0E9BDD]
2016-01-14 17:16:25,769 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:25,770 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791785566_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791785566_0001_1_00, startRequestedTime=1452791785769, startedTime=1452791785769
2016-01-14 17:16:25,770 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791785566_0001_1_00 [B82796B6922744D188B575C1EA0E9BDD] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:25,770 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:25,771 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791785566_0001_1][Event:TASK_STARTED]: vertexName=B82796B6922744D188B575C1EA0E9BDD, taskId=task_1452791785566_0001_1_00_000000, scheduledTime=1452791785770, launchTime=1452791785770
2016-01-14 17:16:25,771 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791785566_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:25,771 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791785566_0001_1_00 for vertex: vertex_1452791785566_0001_1_01 [A03ACB1ABF6542DDBDEDB2A899735A26] numStartedSources: 1 numSources: 1
2016-01-14 17:16:25,771 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791785566_0001_1_01 [A03ACB1ABF6542DDBDEDB2A899735A26] with distanceFromRoot: 1
2016-01-14 17:16:25,771 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791785566_0001_1_00 [B82796B6922744D188B575C1EA0E9BDD]
2016-01-14 17:16:25,772 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: B82796B6922744D188B575C1EA0E9BDD in vertex: A03ACB1ABF6542DDBDEDB2A899735A26
2016-01-14 17:16:25,772 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: A03ACB1ABF6542DDBDEDB2A899735A26 with 1 source tasks and 1 pending tasks
2016-01-14 17:16:25,772 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:25,772 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791785566_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791785566_0001_1_01, startRequestedTime=1452791785771, startedTime=1452791785771
2016-01-14 17:16:25,772 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791785566_0001_1_01 [A03ACB1ABF6542DDBDEDB2A899735A26] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:25,773 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [2278A8A555414C9987A3DB580078E1A0/F1016629791B47B196AB1E6754DFB850] every/(1/1) ...atform/-1_UNKNOWN_REPLACE, VertexName: B82796B6922744D188B575C1EA0E9BDD, VertexParallelism: 1, TaskAttemptID:attempt_1452791785566_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C68FF5DFB480473F9C07824022A7DEAF, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A03ACB1ABF6542DDBDEDB2A899735A26, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:25,773 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791785566_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:25,773 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:25,774 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:25,774 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:25,774 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:25,775 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791785566_0001_00_000001
2016-01-14 17:16:25,775 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791785566_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:25,776 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791785566_0001_00_000001
2016-01-14 17:16:25,777 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:25,777 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791785566_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:25,777 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:25,778 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791785566_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791785566_0001_00_000001, launchTime=1452791785777
2016-01-14 17:16:25,778 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791785566_0001_00_000001
2016-01-14 17:16:25,779 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791785566_0001_1_00_000000_0] to container: [container_1452791785566_0001_00_000001]
2016-01-14 17:16:25,779 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791785566_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:25,779 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791785566_0001_00_000001 given task: attempt_1452791785566_0001_1_00_000000_0
2016-01-14 17:16:25,780 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791785566_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791785566_0001_1_00_000000_0
2016-01-14 17:16:25,780 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791785566_0001_1_00_000000_0] started. Is using containerId: [container_1452791785566_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:25,780 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791785566_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B82796B6922744D188B575C1EA0E9BDD, taskAttemptId=attempt_1452791785566_0001_1_00_000000_0, startTime=1452791785779, containerId=container_1452791785566_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791785566_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:25,780 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791785566_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:25,781 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791785566_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:25,780 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:25 Starting to run new task attempt: attempt_1452791785566_0001_1_00_000000_0
2016-01-14 17:16:25,781 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791785566_0001_1_00_000000_0
2016-01-14 17:16:25,781 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:25,782 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:25,782 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:25,782 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:25,782 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:25,783 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:25,783 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [2278A8A555414C9987A3DB580078E1A0/F1016629791B47B196AB1E6754DFB850] every/(1/1) ...atform/-1_UNKNOWN_REPLACE, VertexName: B82796B6922744D188B575C1EA0E9BDD, VertexParallelism: 1, TaskAttemptID:attempt_1452791785566_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C68FF5DFB480473F9C07824022A7DEAF, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A03ACB1ABF6542DDBDEDB2A899735A26, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:25,783 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:16:25,784 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791785566_0001_1_00_000000_0
2016-01-14 17:16:25,784 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:25,784 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:25,785 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C68FF5DFB480473F9C07824022A7DEAF, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:25,785 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:25,785 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C68FF5DFB480473F9C07824022A7DEAF
2016-01-14 17:16:25,786 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:25,786 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: C68FF5DFB480473F9C07824022A7DEAF
2016-01-14 17:16:25,785 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791785566_0001, with dag id: dag_1452791785566_0001_1
2016-01-14 17:16:25,787 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791785566_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:25,790 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:25,793 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:25,793 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: C68FF5DFB480473F9C07824022A7DEAF
2016-01-14 17:16:25,793 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C68FF5DFB480473F9C07824022A7DEAF
2016-01-14 17:16:25,795 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A03ACB1ABF6542DDBDEDB2A899735A26, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:25,795 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:25,796 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A03ACB1ABF6542DDBDEDB2A899735A26
2016-01-14 17:16:25,798 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B82796B6922744D188B575C1EA0E9BDD, ordinal: 0
2016-01-14 17:16:25,798 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B82796B6922744D188B575C1EA0E9BDD, mem on start (mb), free: 245, total: 490, max: 672
2016-01-14 17:16:25,798 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:25,799 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:25,799 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:25,799 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:25,799 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: A03ACB1ABF6542DDBDEDB2A899735A26
2016-01-14 17:16:25,799 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A03ACB1ABF6542DDBDEDB2A899735A26
2016-01-14 17:16:25,800 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:25,800 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:25,800 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:25,800 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:25,800 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:25,800 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:25,802 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 17:16:25,802 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C68FF5DFB480473F9C07824022A7DEAF, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:25,802 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A03ACB1ABF6542DDBDEDB2A899735A26, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:25,802 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:25,803 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C68FF5DFB480473F9C07824022A7DEAF being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:25,803 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:25,805 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:25,803 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C68FF5DFB480473F9C07824022A7DEAF
2016-01-14 17:16:25,806 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:25,806 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C68FF5DFB480473F9C07824022A7DEAF
2016-01-14 17:16:25,806 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:25,806 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791785566_0001_1_00_000000_0
2016-01-14 17:16:25,809 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:25,809 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: C68FF5DFB480473F9C07824022A7DEAF
2016-01-14 17:16:25,809 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 655B258C0C514A82B75762213B927239
2016-01-14 17:16:25,809 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 655B258C0C514A82B75762213B927239
2016-01-14 17:16:25,810 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@248c1402; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@6875a20c; comparator=cascading.tuple.hadoop.util.TupleComparator@2e1f768; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:25,810 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:25,811 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:25,811 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:25,811 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:25,815 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@e85eef9
2016-01-14 17:16:25,818 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@e85eef9
2016-01-14 17:16:25,819 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:25,819 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:25,826 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:25,826 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:25,826 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:25,827 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:25,827 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:25,828 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:25,828 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:25,829 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B82796B6922744D188B575C1EA0E9BDD, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:25,831 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791785566_0001_1_00_000000_0
2016-01-14 17:16:25,832 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B82796B6922744D188B575C1EA0E9BDD, mem on close (mb), free: 143, total: 490, max: 672
2016-01-14 17:16:25,832 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:25,832 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:25,832 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:25,832 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:25,835 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:25 Completed running task attempt: attempt_1452791785566_0001_1_00_000000_0
2016-01-14 17:16:25,838 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791785566_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:25,839 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791785566_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791785566_0001_1_00, eventCount=1
2016-01-14 17:16:25,840 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:25,840 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791785566_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B82796B6922744D188B575C1EA0E9BDD, taskAttemptId=attempt_1452791785566_0001_1_00_000000_0, startTime=1452791785779, finishTime=1452791785840, timeTaken=61, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=513802240, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791785829, Process_Duration=2, Process_End_Time=1452791785831, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:25,840 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791785566_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:25,840 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791785566_0001_1_00_000000_0
2016-01-14 17:16:25,840 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:25,841 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791785566_0001_1][Event:TASK_FINISHED]: vertexName=B82796B6922744D188B575C1EA0E9BDD, taskId=task_1452791785566_0001_1_00_000000, startTime=1452791785779, finishTime=1452791785841, timeTaken=62, status=SUCCEEDED, successfulAttemptID=attempt_1452791785566_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=513802240, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791785829, Process_Duration=2, Process_End_Time=1452791785831, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:25,841 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:25,842 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791785566_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:25,842 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791785566_0001_1_00 [B82796B6922744D188B575C1EA0E9BDD] : 1
2016-01-14 17:16:25,842 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791785566_0001_1_00 [B82796B6922744D188B575C1EA0E9BDD]
2016-01-14 17:16:25,844 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:25,844 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791785566_0001_1][Event:VERTEX_FINISHED]: vertexName=B82796B6922744D188B575C1EA0E9BDD, vertexId=vertex_1452791785566_0001_1_00, initRequestedTime=1452791785755, initedTime=1452791785768, startRequestedTime=1452791785769, startedTime=1452791785769, finishTime=1452791785842, timeTaken=73, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=513802240, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791785829, Process_Duration=2, Process_End_Time=1452791785831, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791785779, firstTasksToStart=[ task_1452791785566_0001_1_00_000000 ], lastTaskFinishTime=1452791785840, lastTasksToFinish=[ task_1452791785566_0001_1_00_000000 ], minTaskDuration=61, maxTaskDuration=61, avgTaskDuration=61.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791785566_0001_1_00_000000 ], longestDurationTasks=[ task_1452791785566_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:25,844 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791785566_0001_1_00 [B82796B6922744D188B575C1EA0E9BDD] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:25,845 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791785566_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:25,845 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=513802240, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791785829, Process_Duration=2, Process_End_Time=1452791785831, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:25,845 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791785566_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791785566_0001_00_000001, stoppedTime=1452791785845, exitStatus=0
2016-01-14 17:16:25,845 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:25,845 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:25,845 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:25,845 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791785566_0001_00_000001
2016-01-14 17:16:25,845 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791785566_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:25,846 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791785566_0001_1_01 [A03ACB1ABF6542DDBDEDB2A899735A26] attempt: attempt_1452791785566_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:25,846 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: A03ACB1ABF6542DDBDEDB2A899735A26
2016-01-14 17:16:25,846 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791785566_0001_1_01 [A03ACB1ABF6542DDBDEDB2A899735A26]
2016-01-14 17:16:25,847 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791785566_0001_1_00 [B82796B6922744D188B575C1EA0E9BDD] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:25,847 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791785566_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:25,846 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791785566_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:25,848 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791785566_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:25,848 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791785566_0001_00_000001, Exiting
2016-01-14 17:16:25,847 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791785566_0001_1][Event:TASK_STARTED]: vertexName=A03ACB1ABF6542DDBDEDB2A899735A26, taskId=task_1452791785566_0001_1_01_000000, scheduledTime=1452791785847, launchTime=1452791785847
2016-01-14 17:16:25,847 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:25,848 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791785566_0001_00_000001
2016-01-14 17:16:25,848 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791785566_0001_00_000001
2016-01-14 17:16:25,848 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791785566_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:25,850 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791785566_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:25,852 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791785566_0001_1_01 [A03ACB1ABF6542DDBDEDB2A899735A26]
2016-01-14 17:16:25,853 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791785566_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:25,853 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791785566_0001_00_000001 completed successfully
2016-01-14 17:16:25,853 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [2278A8A555414C9987A3DB580078E1A0/F1016629791B47B196AB1E6754DFB850] every/(1/1) ...atform/-1_UNKNOWN_REPLACE, VertexName: A03ACB1ABF6542DDBDEDB2A899735A26, VertexParallelism: 1, TaskAttemptID:attempt_1452791785566_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B82796B6922744D188B575C1EA0E9BDD, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=8A4B00A77F974BF6839CC3C405CF60A9, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:25,854 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791785566_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:25,854 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791785566_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:25,854 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:25,855 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791785566_0001_00_000002
2016-01-14 17:16:25,855 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791785566_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:25,857 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791785566_0001_00_000002
2016-01-14 17:16:25,857 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 17:16:25,860 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:25,860 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791785566_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:25,860 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791785566_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791785566_0001_00_000002, launchTime=1452791785860
2016-01-14 17:16:25,861 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791785566_0001_00_000002
2016-01-14 17:16:25,861 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791785566_0001_1_01_000000_0] to container: [container_1452791785566_0001_00_000002]
2016-01-14 17:16:25,861 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791785566_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:25,861 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791785566_0001_00_000002 given task: attempt_1452791785566_0001_1_01_000000_0
2016-01-14 17:16:25,862 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791785566_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791785566_0001_1_01_000000_0
2016-01-14 17:16:25 Starting to run new task attempt: attempt_1452791785566_0001_1_01_000000_0
2016-01-14 17:16:25,862 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791785566_0001_1_01_000000_0] started. Is using containerId: [container_1452791785566_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:25,862 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791785566_0001_1_01_000000_0
2016-01-14 17:16:25,863 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:25,863 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791785566_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A03ACB1ABF6542DDBDEDB2A899735A26, taskAttemptId=attempt_1452791785566_0001_1_01_000000_0, startTime=1452791785862, containerId=container_1452791785566_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791785566_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:25,863 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:25,863 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:25,863 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791785566_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:25,863 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:25,864 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791785566_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:25,864 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:25,864 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:25,864 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:25,865 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [2278A8A555414C9987A3DB580078E1A0/F1016629791B47B196AB1E6754DFB850] every/(1/1) ...atform/-1_UNKNOWN_REPLACE, VertexName: A03ACB1ABF6542DDBDEDB2A899735A26, VertexParallelism: 1, TaskAttemptID:attempt_1452791785566_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B82796B6922744D188B575C1EA0E9BDD, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=8A4B00A77F974BF6839CC3C405CF60A9, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:25,865 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:16:25,865 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791785566_0001_1_01_000000_0
2016-01-14 17:16:25,866 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:25,866 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:25,866 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791785566_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:25,867 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B82796B6922744D188B575C1EA0E9BDD, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:25,867 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=8A4B00A77F974BF6839CC3C405CF60A9, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:25,867 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:25,867 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:25,868 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B82796B6922744D188B575C1EA0E9BDD
2016-01-14 17:16:25,868 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:25,869 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 8A4B00A77F974BF6839CC3C405CF60A9
2016-01-14 17:16:25,869 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:25,869 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 8A4B00A77F974BF6839CC3C405CF60A9
2016-01-14 17:16:25,869 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:25,870 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:25,870 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: B82796B6922744D188B575C1EA0E9BDD
2016-01-14 17:16:25,871 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B82796B6922744D188B575C1EA0E9BDD
2016-01-14 17:16:25,872 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:25,882 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A03ACB1ABF6542DDBDEDB2A899735A26, ordinal: 1
2016-01-14 17:16:25,882 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A03ACB1ABF6542DDBDEDB2A899735A26, mem on start (mb), free: 137, total: 490, max: 672
2016-01-14 17:16:25,882 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:25,882 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:25,883 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:25,883 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:25,883 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 8A4B00A77F974BF6839CC3C405CF60A9
2016-01-14 17:16:25,884 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:25,884 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:25,884 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:25,885 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:25,885 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:25,885 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:25,886 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:25,886 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 8A4B00A77F974BF6839CC3C405CF60A9, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:25,886 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B82796B6922744D188B575C1EA0E9BDD, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 17:16:25,886 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:25,887 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B82796B6922744D188B575C1EA0E9BDD
2016-01-14 17:16:25,889 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:25,889 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:25,889 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: B82796B6922744D188B575C1EA0E9BDD with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:25,889 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 17:16:25,889 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:25,890 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:25,890 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B82796B6922744D188B575C1EA0E9BDD being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:25,890 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B82796B6922744D188B575C1EA0E9BDD
2016-01-14 17:16:25,893 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:25,894 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:25,894 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:25,894 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791785566_0001_1_01_000000_0
2016-01-14 17:16:25,895 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791785566_0001_1_00_000000_0_10051, runDuration: 40234, ]
2016-01-14 17:16:25,898 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791785566_0001_1_00_000000_0_10051] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791785898, TimeTaken=2, Rate=0.14 MB/s
2016-01-14 17:16:25,898 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : B82796B6922744D188B575C1EA0E9BDD
2016-01-14 17:16:25,898 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:25,898 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [B82796B6922744D188B575C1EA0E9BDD] #9 in 2ms
2016-01-14 17:16:25,898 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [B82796B6922744D188B575C1EA0E9BDD] #9, status:false, isInterrupted:false
2016-01-14 17:16:25,898 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:25,899 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:25,900 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:25,900 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:25,900 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:25,900 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 655B258C0C514A82B75762213B927239
2016-01-14 17:16:25,900 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:25,901 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : B82796B6922744D188B575C1EA0E9BDD
2016-01-14 17:16:25,901 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:25,900 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_REPLACE"], id: 8A4B00A77F974BF6839CC3C405CF60A9
2016-01-14 17:16:25,901 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_REPLACE"]
2016-01-14 17:16:25,902 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 655B258C0C514A82B75762213B927239, for 1 inputs
2016-01-14 17:16:25,902 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A03ACB1ABF6542DDBDEDB2A899735A26, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:25,903 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@19e0e2f5; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@7b545018
2016-01-14 17:16:25,905 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:25,909 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(92)) - caught throwable
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:25,911 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:25,912 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:25,912 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791785566_0001_1_01_000000_0
2016-01-14 17:16:25,912 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791785566_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:25,913 INFO  task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1452791785566_0001_1_01_000000_0
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:25,919 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=528, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=513802240, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791785902, Process_Duration=11, Process_End_Time=1452791785913, Read_Duration=1, Tuples_Read=1]]
2016-01-14 17:16:25,919 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A03ACB1ABF6542DDBDEDB2A899735A26, mem on close (mb), free: 133, total: 490, max: 672
2016-01-14 17:16:25,919 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=A03ACB1ABF6542DDBDEDB2A899735A26, index=1
2016-01-14 17:16:25,922 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:25,922 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791785566_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A03ACB1ABF6542DDBDEDB2A899735A26, taskAttemptId=attempt_1452791785566_0001_1_01_000000_0, startTime=1452791785862, finishTime=1452791785919, timeTaken=57, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=528, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=513802240, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791785902, Process_Duration=11, Process_End_Time=1452791785913, Read_Duration=1, Tuples_Read=1
2016-01-14 17:16:25,922 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791785566_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-01-14 17:16:25,923 INFO  impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1452791785566_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-01-14 17:16:25,923 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791785566_0001_1][Event:TASK_FINISHED]: vertexName=A03ACB1ABF6542DDBDEDB2A899735A26, taskId=task_1452791785566_0001_1_01_000000, startTime=1452791785862, finishTime=1452791785923, timeTaken=61, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
], counters=Counters: 0
2016-01-14 17:16:25,924 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:25,924 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791785566_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-01-14 17:16:25,924 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791785566_0001_1_01 [A03ACB1ABF6542DDBDEDB2A899735A26] : 1
2016-01-14 17:16:25,924 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:25,924 INFO  impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1452791785566_0001_1_01 [A03ACB1ABF6542DDBDEDB2A899735A26] because task failed: task_1452791785566_0001_1_01_000000
2016-01-14 17:16:25,925 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: B82796B6922744D188B575C1EA0E9BDD
2016-01-14 17:16:25,925 INFO  impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1452791785566_0001_1_01 [A03ACB1ABF6542DDBDEDB2A899735A26] due to trigger: OWN_TASK_FAILURE
2016-01-14 17:16:25,925 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:25,926 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:25,926 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-01-14 17:16:25,926 INFO  impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1452791785566_0001_1_01 [A03ACB1ABF6542DDBDEDB2A899735A26]
2016-01-14 17:16:25,926 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:25,927 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:25,927 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:25,927 INFO  impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=8A4B00A77F974BF6839CC3C405CF60A9, vertexId=vertex_1452791785566_0001_1_01 [A03ACB1ABF6542DDBDEDB2A899735A26]
2016-01-14 17:16:25,927 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=A03ACB1ABF6542DDBDEDB2A899735A26, sourceVertex=B82796B6922744D188B575C1EA0E9BDD
2016-01-14 17:16:25,928 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=A03ACB1ABF6542DDBDEDB2A899735A26, sourceVertex=8A4B00A77F974BF6839CC3C405CF60A9
2016-01-14 17:16:25,928 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791785566_0001_00_000002
2016-01-14 17:16:25,928 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791785566_0001_00_000002
2016-01-14 17:16:25,931 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791785566_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-01-14 17:16:25,931 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1452791785566_0001_00_000002 completed but with errors
2016-01-14 17:16:25,933 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791785566_0001_1][Event:VERTEX_FINISHED]: vertexName=A03ACB1ABF6542DDBDEDB2A899735A26, vertexId=vertex_1452791785566_0001_1_01, initRequestedTime=1452791785757, initedTime=1452791785765, startRequestedTime=1452791785771, startedTime=1452791785771, finishTime=1452791785933, timeTaken=162, status=FAILED, diagnostics=Task failed, taskId=task_1452791785566_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1452791785566_0001_1_01 [A03ACB1ABF6542DDBDEDB2A899735A26] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1452791785862, firstTasksToStart=[ task_1452791785566_0001_1_01_000000 ], lastTaskFinishTime=1452791785919, lastTasksToFinish=[ task_1452791785566_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[  ], longestDurationTasks=[  ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-01-14 17:16:25,933 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:25,934 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791785566_0001_1_01 [A03ACB1ABF6542DDBDEDB2A899735A26] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-01-14 17:16:25,934 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:25,935 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791785566_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791785566_0001_00_000002, stoppedTime=1452791785934, exitStatus=0
2016-01-14 17:16:25,935 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791785566_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:25,935 INFO  node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1452791785566_0001_1_01_000000_0 failed: true container: container_1452791785566_0001_00_000002 numFailedTAs: 0
2016-01-14 17:16:25,936 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1452791785566_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-01-14 17:16:25,936 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791785566_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:16:25,937 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791785566_0001_1_01 [A03ACB1ABF6542DDBDEDB2A899735A26] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-01-14 17:16:25,937 INFO  impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-01-14 17:16:25,938 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791785566_0001_1
2016-01-14 17:16:25,938 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1452791785566_0001_1_00 [B82796B6922744D188B575C1EA0E9BDD]
2016-01-14 17:16:25,938 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 8A4B00A77F974BF6839CC3C405CF60A9 for vertex: vertex_1452791785566_0001_1_01 [A03ACB1ABF6542DDBDEDB2A899735A26]
2016-01-14 17:16:25,941 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791785566_0001_1, queueSize=0
2016-01-14 17:16:25,942 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:25,943 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791785566_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791785566_0001_1, startTime=1452791785741, finishTime=1452791785939, timeTaken=198, status=FAILED, diagnostics=Vertex failed, vertexName=A03ACB1ABF6542DDBDEDB2A899735A26, vertexId=vertex_1452791785566_0001_1_01, diagnostics=[Task failed, taskId=task_1452791785566_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1452791785566_0001_1_01 [A03ACB1ABF6542DDBDEDB2A899735A26] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=513802240, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791785829, Process_Duration=2, Process_End_Time=1452791785831, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:25,943 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791785566_0001_1 finished with state: FAILED
2016-01-14 17:16:25,943 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791785566_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:25,943 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452791785566_0001_00_000002
2016-01-14 17:16:25,944 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791785566_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-01-14 17:16:25 Completed Dag: dag_1452791785566_0001_1
2016-01-14 17:16:25,944 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791785566_0001_1, dagState=FAILED
2016-01-14 17:16:25,945 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791785566_0001_1_post
2016-01-14 17:16:25,945 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:25,945 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:25,949 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...atform/-1_UNKNOWN_REPLACE, applicationId=application_1452791785566_0001
2016-01-14 17:16:25,951 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:25,951 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:25,951 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:25,951 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:25,952 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:25,952 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:25,952 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:25,953 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:25,953 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:25,953 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:25,954 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:25,954 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:25,954 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:25,960 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/2278A8A555414C9987A3DB580078E1A0/F1016629791B47B196AB1E6754DFB850/.tez/application_1452791785566_0001
2016-01-14 17:16:25,960 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping all jobs
2016-01-14 17:16:25,960 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping: (1/1) ...atform/-1_UNKNOWN_REPLACE
2016-01-14 17:16:25,962 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopped all jobs
2016-01-14 17:16:25,963 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.416
2016-01-14 17:16:25,963 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_UNKNOWN_REPLACE
2016-01-14 17:16:25,991 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:25,992 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:26,031 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.016
2016-01-14 17:16:26,054 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.039
2016-01-14 17:16:26,055 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:26,055 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:26,056 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:26,078 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:26,090 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_SWAP
2016-01-14 17:16:26,165 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:26,165 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:26,166 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_SWAP"]
2016-01-14 17:16:26,166 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:26,167 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:26,167 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:26,168 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...yplatform/-1_UNKNOWN_SWAP
2016-01-14 17:16:26,169 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:26,181 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:26,186 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:26,187 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:26,187 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:26,200 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/399A2353C8D447EFAE57D50D3ED33D46/CBFB840D95BE4B8E9769463E4B40B4C7/.tez/application_1452791786186_0001 doesn't exist and is created
2016-01-14 17:16:26,225 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:26,225 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:26,226 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/399A2353C8D447EFAE57D50D3ED33D46/CBFB840D95BE4B8E9769463E4B40B4C7/.tez/application_1452791786186_0001
2016-01-14 17:16:26,232 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791786186_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:26,248 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:26,249 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:26,251 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:26,252 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:26,252 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/399A2353C8D447EFAE57D50D3ED33D46/CBFB840D95BE4B8E9769463E4B40B4C7/.tez/application_1452791786186_0001/localmode-log-dir
2016-01-14 17:16:26,253 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/399A2353C8D447EFAE57D50D3ED33D46/CBFB840D95BE4B8E9769463E4B40B4C7/.tez/application_1452791786186_0001/localmode-log-dir/history.txt.appattempt_1452791786186_0001_000000, maxErrors=10
2016-01-14 17:16:26,253 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:26,253 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791786186_0001_000000, appSubmitTime=1452791786231, launchTime=1452791786231
2016-01-14 17:16:26,258 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:26,259 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:26,261 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:26,262 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:26,265 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:60497
2016-01-14 17:16:26,276 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:26,280 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791786186_0001_000000, startTime=1452791786280
2016-01-14 17:16:26,280 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:26,280 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:26,326 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:26,326 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:26,326 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...yplatform/-1_UNKNOWN_SWAP, applicationId=application_1452791786186_0001, dagName=[399A2353C8D447EFAE57D50D3ED33D46/CBFB840D95BE4B8E9769463E4B40B4C7] every/(1/1) ...yplatform/-1_UNKNOWN_SWAP
2016-01-14 17:16:26,344 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [399A2353C8D447EFAE57D50D3ED33D46/CBFB840D95BE4B8E9769463E4B40B4C7] every/(1/1) ...yplatform/-1_UNKNOWN_SWAP
2016-01-14 17:16:26,345 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791786186_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/399A2353C8D447EFAE57D50D3ED33D46/CBFB840D95BE4B8E9769463E4B40B4C7/.tez/application_1452791786186_0001/localmode-log-dir/dag_1452791786186_0001_1.dot
2016-01-14 17:16:26,346 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/399A2353C8D447EFAE57D50D3ED33D46/CBFB840D95BE4B8E9769463E4B40B4C7/.tez/application_1452791786186_0001/localmode-log-dir/dag_1452791786186_0001_1-tez-dag.pb.txt
2016-01-14 17:16:26,353 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791786186_0001_1
2016-01-14 17:16:26,353 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:26,353 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [399A2353C8D447EFAE57D50D3ED33D46/CBFB840D95BE4B8E9769463E4B40B4C7] every/(1/1) ...yplatform/-1_UNKNOWN_SWAP
2016-01-14 17:16:26 Running Dag: dag_1452791786186_0001_1
2016-01-14 17:16:26,367 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:26,368 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791786186_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791786186_0001_1, submitTime=1452791786344
2016-01-14 17:16:26,368 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex E48A74DAC465470382CB4BF3D9B3D025
2016-01-14 17:16:26,369 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:26,370 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:26,370 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791786186_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791786186_0001_1, initTime=1452791786368
2016-01-14 17:16:26,371 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791786186_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:26,371 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...yplatform/-1_UNKNOWN_SWAP, applicationId=application_1452791786186_0001, dagName=[399A2353C8D447EFAE57D50D3ED33D46/CBFB840D95BE4B8E9769463E4B40B4C7] every/(1/1) ...yplatform/-1_UNKNOWN_SWAP
2016-01-14 17:16:26,372 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791786186_0001_1][Event:DAG_STARTED]: dagID=dag_1452791786186_0001_1, startTime=1452791786372
2016-01-14 17:16:26,372 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:26,372 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:26,372 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791786186_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:26,372 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: F2542EDAE8564AC6BC9AD993FBE8C96B : {48F70E2FE492487CB21CF880571CA1E0={InputName=48F70E2FE492487CB21CF880571CA1E0}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:26,372 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 48F70E2FE492487CB21CF880571CA1E0, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:26,373 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791786186_0001_1_00 [F2542EDAE8564AC6BC9AD993FBE8C96B]
2016-01-14 17:16:26,373 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791786186_0001_1_00 [F2542EDAE8564AC6BC9AD993FBE8C96B]
2016-01-14 17:16:26,373 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791786186_0001_1_00 [F2542EDAE8564AC6BC9AD993FBE8C96B]
2016-01-14 17:16:26,373 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791786186_0001_1_00 [F2542EDAE8564AC6BC9AD993FBE8C96B]. Starting root input initializers: 1
2016-01-14 17:16:26,374 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791786186_0001_1_00 [F2542EDAE8564AC6BC9AD993FBE8C96B] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:26,374 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791786186_0001_1_01 [E48A74DAC465470382CB4BF3D9B3D025]
2016-01-14 17:16:26,376 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 48F70E2FE492487CB21CF880571CA1E0 on vertex vertex_1452791786186_0001_1_00 [F2542EDAE8564AC6BC9AD993FBE8C96B]
2016-01-14 17:16:26,377 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:26,379 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791786186_0001_1_01 [E48A74DAC465470382CB4BF3D9B3D025]
2016-01-14 17:16:26,380 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791786186_0001_1_01 [E48A74DAC465470382CB4BF3D9B3D025]
2016-01-14 17:16:26,380 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791786186_0001_1_01 [E48A74DAC465470382CB4BF3D9B3D025]
2016-01-14 17:16:26,381 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=27FF6E394A7248CD83546677F4B0BBDD, vertexId=vertex_1452791786186_0001_1_01 [E48A74DAC465470382CB4BF3D9B3D025], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:26,381 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=27FF6E394A7248CD83546677F4B0BBDD, vertexId=vertex_1452791786186_0001_1_01 [E48A74DAC465470382CB4BF3D9B3D025]
2016-01-14 17:16:26,378 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:26,383 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 48F70E2FE492487CB21CF880571CA1E0 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:26,385 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:26,385 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:26,385 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=27FF6E394A7248CD83546677F4B0BBDD, vertexName=E48A74DAC465470382CB4BF3D9B3D025, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:26,386 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=27FF6E394A7248CD83546677F4B0BBDD, vertexId=vertex_1452791786186_0001_1_01 [E48A74DAC465470382CB4BF3D9B3D025]
2016-01-14 17:16:26,389 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:26,390 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791786186_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E48A74DAC465470382CB4BF3D9B3D025, vertexId=vertex_1452791786186_0001_1_01, initRequestedTime=1452791786374, initedTime=1452791786389, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:26,390 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791786186_0001_1_01 [E48A74DAC465470382CB4BF3D9B3D025] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:26,391 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:26,392 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 48F70E2FE492487CB21CF880571CA1E0 on vertex vertex_1452791786186_0001_1_00 [F2542EDAE8564AC6BC9AD993FBE8C96B]
2016-01-14 17:16:26,393 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791786186_0001_1_00 [F2542EDAE8564AC6BC9AD993FBE8C96B] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:26,393 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {48F70E2FE492487CB21CF880571CA1E0=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:26,393 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791786186_0001_1_00 [F2542EDAE8564AC6BC9AD993FBE8C96B] parallelism set to 1
2016-01-14 17:16:26,393 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791786186_0001_1_00 [F2542EDAE8564AC6BC9AD993FBE8C96B] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:26,394 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791786186_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791786186_0001_1_00, eventCount=1
2016-01-14 17:16:26,396 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:26,396 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791786186_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F2542EDAE8564AC6BC9AD993FBE8C96B, vertexId=vertex_1452791786186_0001_1_00, initRequestedTime=1452791786372, initedTime=1452791786394, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:26,396 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791786186_0001_1_00 [F2542EDAE8564AC6BC9AD993FBE8C96B] with distanceFromRoot: 0
2016-01-14 17:16:26,397 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791786186_0001_1_00 [F2542EDAE8564AC6BC9AD993FBE8C96B] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:26,397 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in F2542EDAE8564AC6BC9AD993FBE8C96B
2016-01-14 17:16:26,397 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791786186_0001_1_00 [F2542EDAE8564AC6BC9AD993FBE8C96B]
2016-01-14 17:16:26,398 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:26,398 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791786186_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791786186_0001_1_00, startRequestedTime=1452791786397, startedTime=1452791786397
2016-01-14 17:16:26,399 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791786186_0001_1_00 [F2542EDAE8564AC6BC9AD993FBE8C96B] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:26,399 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:26,400 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791786186_0001_1][Event:TASK_STARTED]: vertexName=F2542EDAE8564AC6BC9AD993FBE8C96B, taskId=task_1452791786186_0001_1_00_000000, scheduledTime=1452791786399, launchTime=1452791786399
2016-01-14 17:16:26,400 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791786186_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:26,400 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791786186_0001, with dag id: dag_1452791786186_0001_1
2016-01-14 17:16:26,400 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791786186_0001_1_00 for vertex: vertex_1452791786186_0001_1_01 [E48A74DAC465470382CB4BF3D9B3D025] numStartedSources: 1 numSources: 1
2016-01-14 17:16:26,401 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791786186_0001_1_01 [E48A74DAC465470382CB4BF3D9B3D025] with distanceFromRoot: 1
2016-01-14 17:16:26,402 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791786186_0001_1_00 [F2542EDAE8564AC6BC9AD993FBE8C96B]
2016-01-14 17:16:26,403 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: F2542EDAE8564AC6BC9AD993FBE8C96B in vertex: E48A74DAC465470382CB4BF3D9B3D025
2016-01-14 17:16:26,403 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: E48A74DAC465470382CB4BF3D9B3D025 with 1 source tasks and 1 pending tasks
2016-01-14 17:16:26,404 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:26,404 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791786186_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791786186_0001_1_01, startRequestedTime=1452791786403, startedTime=1452791786403
2016-01-14 17:16:26,404 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791786186_0001_1_01 [E48A74DAC465470382CB4BF3D9B3D025] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:26,405 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [399A2353C8D447EFAE57D50D3ED33D46/CBFB840D95BE4B8E9769463E4B40B4C7] every/(1/1) ...yplatform/-1_UNKNOWN_SWAP, VertexName: F2542EDAE8564AC6BC9AD993FBE8C96B, VertexParallelism: 1, TaskAttemptID:attempt_1452791786186_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=48F70E2FE492487CB21CF880571CA1E0, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E48A74DAC465470382CB4BF3D9B3D025, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:26,405 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791786186_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:26,405 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:26,406 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:26,407 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:26,407 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:26,408 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791786186_0001_00_000001
2016-01-14 17:16:26,408 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791786186_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:26,409 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791786186_0001_00_000001
2016-01-14 17:16:26,409 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:26,410 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791786186_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:26,411 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:26,411 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791786186_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791786186_0001_00_000001, launchTime=1452791786410
2016-01-14 17:16:26,412 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791786186_0001_00_000001
2016-01-14 17:16:26,415 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791786186_0001_1_00_000000_0] to container: [container_1452791786186_0001_00_000001]
2016-01-14 17:16:26,415 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791786186_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:26,415 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791786186_0001_00_000001 given task: attempt_1452791786186_0001_1_00_000000_0
2016-01-14 17:16:26,415 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791786186_0001_1_00_000000_0] started. Is using containerId: [container_1452791786186_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:26,416 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791786186_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F2542EDAE8564AC6BC9AD993FBE8C96B, taskAttemptId=attempt_1452791786186_0001_1_00_000000_0, startTime=1452791786415, containerId=container_1452791786186_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791786186_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:26,416 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791786186_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:26,416 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791786186_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:26,416 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:26,416 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791786186_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791786186_0001_1_00_000000_0
2016-01-14 17:16:26 Starting to run new task attempt: attempt_1452791786186_0001_1_00_000000_0
2016-01-14 17:16:26,419 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791786186_0001_1_00_000000_0
2016-01-14 17:16:26,419 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:26,419 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:26,420 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:26,420 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:26,420 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:26,420 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:26,420 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [399A2353C8D447EFAE57D50D3ED33D46/CBFB840D95BE4B8E9769463E4B40B4C7] every/(1/1) ...yplatform/-1_UNKNOWN_SWAP, VertexName: F2542EDAE8564AC6BC9AD993FBE8C96B, VertexParallelism: 1, TaskAttemptID:attempt_1452791786186_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=48F70E2FE492487CB21CF880571CA1E0, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E48A74DAC465470382CB4BF3D9B3D025, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:26,420 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:16:26,421 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791786186_0001_1_00_000000_0
2016-01-14 17:16:26,421 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:26,421 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:26,423 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=48F70E2FE492487CB21CF880571CA1E0, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:26,423 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:26,423 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 48F70E2FE492487CB21CF880571CA1E0
2016-01-14 17:16:26,424 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 48F70E2FE492487CB21CF880571CA1E0
2016-01-14 17:16:26,426 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:26,426 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=E48A74DAC465470382CB4BF3D9B3D025, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:26,428 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:26,428 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: E48A74DAC465470382CB4BF3D9B3D025
2016-01-14 17:16:26,429 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:26,432 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:26,432 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: E48A74DAC465470382CB4BF3D9B3D025
2016-01-14 17:16:26,438 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:26,438 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 48F70E2FE492487CB21CF880571CA1E0
2016-01-14 17:16:26,438 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 48F70E2FE492487CB21CF880571CA1E0
2016-01-14 17:16:26,439 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: E48A74DAC465470382CB4BF3D9B3D025
2016-01-14 17:16:26,440 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791786186_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:26,443 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F2542EDAE8564AC6BC9AD993FBE8C96B, ordinal: 0
2016-01-14 17:16:26,443 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F2542EDAE8564AC6BC9AD993FBE8C96B, mem on start (mb), free: 54, total: 490, max: 672
2016-01-14 17:16:26,444 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:26,444 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:26,444 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:26,444 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:26,446 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:26,447 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:26,447 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:26,447 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:26,449 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:26,449 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 17:16:26,449 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 48F70E2FE492487CB21CF880571CA1E0, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:26,450 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, E48A74DAC465470382CB4BF3D9B3D025, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:26,450 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:26,450 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 48F70E2FE492487CB21CF880571CA1E0
2016-01-14 17:16:26,450 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:26,450 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 48F70E2FE492487CB21CF880571CA1E0
2016-01-14 17:16:26,451 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 48F70E2FE492487CB21CF880571CA1E0 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:26,451 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:26,451 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:26,452 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:26,452 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791786186_0001_1_00_000000_0
2016-01-14 17:16:26,452 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:26,453 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@79853412
2016-01-14 17:16:26,455 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@79853412
2016-01-14 17:16:26,456 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:26,457 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:26,458 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:26,458 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 48F70E2FE492487CB21CF880571CA1E0
2016-01-14 17:16:26,458 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 72F042DF96BF4F458A63D929E6AEDA94
2016-01-14 17:16:26,459 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 72F042DF96BF4F458A63D929E6AEDA94
2016-01-14 17:16:26,459 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@4cfe7e93; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@58280c62; comparator=cascading.tuple.hadoop.util.TupleComparator@2d452325; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:26,460 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:26,460 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:26,460 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:26,483 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:26,484 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:26,484 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:26,484 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:26,484 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:26,485 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:26,485 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:26,486 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F2542EDAE8564AC6BC9AD993FBE8C96B, all 1 inputs ready in: 00:00:00.001
2016-01-14 17:16:26,489 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791786186_0001_1_00_000000_0
2016-01-14 17:16:26,490 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F2542EDAE8564AC6BC9AD993FBE8C96B, mem on close (mb), free: 155, total: 490, max: 672
2016-01-14 17:16:26,490 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:26,490 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:26,490 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:26,490 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:26,493 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:26 Completed running task attempt: attempt_1452791786186_0001_1_00_000000_0
2016-01-14 17:16:26,497 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791786186_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:26,498 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791786186_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791786186_0001_1_00, eventCount=1
2016-01-14 17:16:26,498 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791786486, Process_Duration=3, Process_End_Time=1452791786489, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:26,498 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791786186_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F2542EDAE8564AC6BC9AD993FBE8C96B, taskAttemptId=attempt_1452791786186_0001_1_00_000000_0, startTime=1452791786415, finishTime=1452791786498, timeTaken=83, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791786486, Process_Duration=3, Process_End_Time=1452791786489, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:26,499 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791786186_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:26,499 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:26,499 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:26,498 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:26,500 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:26,499 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791786186_0001_1_00_000000_0
2016-01-14 17:16:26,501 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791786186_0001_00_000001
2016-01-14 17:16:26,502 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:26,502 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791786186_0001_1][Event:TASK_FINISHED]: vertexName=F2542EDAE8564AC6BC9AD993FBE8C96B, taskId=task_1452791786186_0001_1_00_000000, startTime=1452791786415, finishTime=1452791786501, timeTaken=86, status=SUCCEEDED, successfulAttemptID=attempt_1452791786186_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791786486, Process_Duration=3, Process_End_Time=1452791786489, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:26,502 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791786186_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:26,503 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791786186_0001_1_00 [F2542EDAE8564AC6BC9AD993FBE8C96B] : 1
2016-01-14 17:16:26,503 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791786186_0001_1_00 [F2542EDAE8564AC6BC9AD993FBE8C96B]
2016-01-14 17:16:26,504 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:26,505 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791786186_0001_1][Event:VERTEX_FINISHED]: vertexName=F2542EDAE8564AC6BC9AD993FBE8C96B, vertexId=vertex_1452791786186_0001_1_00, initRequestedTime=1452791786372, initedTime=1452791786394, startRequestedTime=1452791786397, startedTime=1452791786397, finishTime=1452791786503, timeTaken=106, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791786486, Process_Duration=3, Process_End_Time=1452791786489, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791786415, firstTasksToStart=[ task_1452791786186_0001_1_00_000000 ], lastTaskFinishTime=1452791786498, lastTasksToFinish=[ task_1452791786186_0001_1_00_000000 ], minTaskDuration=83, maxTaskDuration=83, avgTaskDuration=83.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791786186_0001_1_00_000000 ], longestDurationTasks=[ task_1452791786186_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:26,506 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791786186_0001_1_00 [F2542EDAE8564AC6BC9AD993FBE8C96B] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:26,506 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791786186_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:26,506 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:26,507 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791786186_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791786186_0001_00_000001, stoppedTime=1452791786506, exitStatus=0
2016-01-14 17:16:26,507 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791786186_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:26,507 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791786186_0001_1_01 [E48A74DAC465470382CB4BF3D9B3D025] attempt: attempt_1452791786186_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:26,507 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: E48A74DAC465470382CB4BF3D9B3D025
2016-01-14 17:16:26,508 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791786186_0001_1_01 [E48A74DAC465470382CB4BF3D9B3D025]
2016-01-14 17:16:26,508 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791786186_0001_1_00 [F2542EDAE8564AC6BC9AD993FBE8C96B] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:26,508 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791786186_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:26,509 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:26,509 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791786186_0001_1][Event:TASK_STARTED]: vertexName=E48A74DAC465470382CB4BF3D9B3D025, taskId=task_1452791786186_0001_1_01_000000, scheduledTime=1452791786508, launchTime=1452791786508
2016-01-14 17:16:26,509 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791786186_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:26,509 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791786186_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:26,510 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791786186_0001_1_01 [E48A74DAC465470382CB4BF3D9B3D025]
2016-01-14 17:16:26,510 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [399A2353C8D447EFAE57D50D3ED33D46/CBFB840D95BE4B8E9769463E4B40B4C7] every/(1/1) ...yplatform/-1_UNKNOWN_SWAP, VertexName: E48A74DAC465470382CB4BF3D9B3D025, VertexParallelism: 1, TaskAttemptID:attempt_1452791786186_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F2542EDAE8564AC6BC9AD993FBE8C96B, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=27FF6E394A7248CD83546677F4B0BBDD, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:26,510 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791786186_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:26,510 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:26,511 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791786186_0001_00_000002
2016-01-14 17:16:26,511 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791786186_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:26,511 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791786186_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:26,512 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791786186_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:26,512 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791786186_0001_00_000001, Exiting
2016-01-14 17:16:26,512 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791786186_0001_00_000001
2016-01-14 17:16:26,512 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791786186_0001_00_000001
2016-01-14 17:16:26,513 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791786186_0001_00_000002
2016-01-14 17:16:26,513 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791786186_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:26,514 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791786186_0001_00_000001 completed successfully
2016-01-14 17:16:26,514 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 17:16:26,514 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 17:16:26,514 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 17:16:26,514 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791786186_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:26,516 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791786186_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:26,516 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:26,517 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791786186_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791786186_0001_00_000002, launchTime=1452791786516
2016-01-14 17:16:26,517 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791786186_0001_00_000002
2016-01-14 17:16:26,518 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791786186_0001_1_01_000000_0] to container: [container_1452791786186_0001_00_000002]
2016-01-14 17:16:26,519 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791786186_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:26,519 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791786186_0001_1_01_000000_0] started. Is using containerId: [container_1452791786186_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:26,519 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791786186_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E48A74DAC465470382CB4BF3D9B3D025, taskAttemptId=attempt_1452791786186_0001_1_01_000000_0, startTime=1452791786519, containerId=container_1452791786186_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791786186_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:26,519 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791786186_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:26,520 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791786186_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:26,520 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:26,520 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791786186_0001_00_000002 given task: attempt_1452791786186_0001_1_01_000000_0
2016-01-14 17:16:26,520 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791786186_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791786186_0001_1_01_000000_0
2016-01-14 17:16:26 Starting to run new task attempt: attempt_1452791786186_0001_1_01_000000_0
2016-01-14 17:16:26,521 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791786186_0001_1_01_000000_0
2016-01-14 17:16:26,521 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:26,522 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:26,522 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:26,522 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:26,522 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:26,522 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:26,522 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [399A2353C8D447EFAE57D50D3ED33D46/CBFB840D95BE4B8E9769463E4B40B4C7] every/(1/1) ...yplatform/-1_UNKNOWN_SWAP, VertexName: E48A74DAC465470382CB4BF3D9B3D025, VertexParallelism: 1, TaskAttemptID:attempt_1452791786186_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F2542EDAE8564AC6BC9AD993FBE8C96B, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=27FF6E394A7248CD83546677F4B0BBDD, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:26,522 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:16:26,523 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791786186_0001_1_01_000000_0
2016-01-14 17:16:26,523 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:26,523 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:26,524 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F2542EDAE8564AC6BC9AD993FBE8C96B, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:26,525 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791786186_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:26,525 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:26,525 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F2542EDAE8564AC6BC9AD993FBE8C96B
2016-01-14 17:16:26,524 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=27FF6E394A7248CD83546677F4B0BBDD, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:26,524 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:26,530 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:26,530 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:26,530 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 27FF6E394A7248CD83546677F4B0BBDD
2016-01-14 17:16:26,531 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:26,531 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 27FF6E394A7248CD83546677F4B0BBDD
2016-01-14 17:16:26,533 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:26,533 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:26,534 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: F2542EDAE8564AC6BC9AD993FBE8C96B
2016-01-14 17:16:26,535 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F2542EDAE8564AC6BC9AD993FBE8C96B
2016-01-14 17:16:26,543 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E48A74DAC465470382CB4BF3D9B3D025, ordinal: 1
2016-01-14 17:16:26,543 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E48A74DAC465470382CB4BF3D9B3D025, mem on start (mb), free: 147, total: 490, max: 672
2016-01-14 17:16:26,544 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:26,544 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:26,544 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:26,546 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:26,546 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 27FF6E394A7248CD83546677F4B0BBDD
2016-01-14 17:16:26,546 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:26,546 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:26,546 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:26,546 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:26,546 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:26,547 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:26,547 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:26,547 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 27FF6E394A7248CD83546677F4B0BBDD, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:26,547 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F2542EDAE8564AC6BC9AD993FBE8C96B, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 17:16:26,547 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:26,548 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F2542EDAE8564AC6BC9AD993FBE8C96B being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:26,548 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F2542EDAE8564AC6BC9AD993FBE8C96B
2016-01-14 17:16:26,548 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:26,548 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:26,548 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:26,548 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:26,549 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: F2542EDAE8564AC6BC9AD993FBE8C96B with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:26,549 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 17:16:26,549 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:26,549 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:26,551 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F2542EDAE8564AC6BC9AD993FBE8C96B
2016-01-14 17:16:26,551 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:26,552 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791786186_0001_1_01_000000_0
2016-01-14 17:16:26,553 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791786186_0001_1_00_000000_0_10057, runDuration: 66609, ]
2016-01-14 17:16:26,554 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:26,555 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791786186_0001_1_00_000000_0_10057] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791786555, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 17:16:26,555 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : F2542EDAE8564AC6BC9AD993FBE8C96B
2016-01-14 17:16:26,555 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 72F042DF96BF4F458A63D929E6AEDA94
2016-01-14 17:16:26,555 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_SWAP"], id: 27FF6E394A7248CD83546677F4B0BBDD
2016-01-14 17:16:26,556 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_SWAP"]
2016-01-14 17:16:26,556 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 72F042DF96BF4F458A63D929E6AEDA94, for 1 inputs
2016-01-14 17:16:26,555 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:26,557 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [F2542EDAE8564AC6BC9AD993FBE8C96B] #10 in 3ms
2016-01-14 17:16:26,557 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [F2542EDAE8564AC6BC9AD993FBE8C96B] #10, status:false, isInterrupted:false
2016-01-14 17:16:26,558 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:26,559 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:26,559 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:26,559 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:26,560 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:26,560 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : F2542EDAE8564AC6BC9AD993FBE8C96B
2016-01-14 17:16:26,560 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:26,560 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E48A74DAC465470382CB4BF3D9B3D025, all 1 inputs ready in: 00:00:00.004
2016-01-14 17:16:26,561 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@40b29eda; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@edb1b65
2016-01-14 17:16:26,564 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:26,565 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:26,565 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791786186_0001_1_01_000000_0
2016-01-14 17:16:26,565 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791786186_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:26,566 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791786186_0001_1_01_000000_0
2016-01-14 17:16:26,566 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E48A74DAC465470382CB4BF3D9B3D025, mem on close (mb), free: 145, total: 490, max: 672
2016-01-14 17:16:26,566 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: F2542EDAE8564AC6BC9AD993FBE8C96B
2016-01-14 17:16:26,566 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:26,567 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:26,567 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:26,567 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:26,567 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:26 Completed running task attempt: attempt_1452791786186_0001_1_01_000000_0
2016-01-14 17:16:26,568 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791786186_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:26,570 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:26,570 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791786186_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E48A74DAC465470382CB4BF3D9B3D025, taskAttemptId=attempt_1452791786186_0001_1_01_000000_0, startTime=1452791786519, finishTime=1452791786569, timeTaken=50, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=187, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791786560, Process_Duration=6, Process_End_Time=1452791786566, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:26,570 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791786186_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:26,570 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791786186_0001_1_01_000000_0
2016-01-14 17:16:26,571 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791786186_0001_1][Event:TASK_FINISHED]: vertexName=E48A74DAC465470382CB4BF3D9B3D025, taskId=task_1452791786186_0001_1_01_000000, startTime=1452791786519, finishTime=1452791786570, timeTaken=51, status=SUCCEEDED, successfulAttemptID=attempt_1452791786186_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=187, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791786560, Process_Duration=6, Process_End_Time=1452791786566, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:26,571 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:26,571 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:26,571 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791786186_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:26,572 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791786186_0001_1_01 [E48A74DAC465470382CB4BF3D9B3D025] : 1
2016-01-14 17:16:26,572 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791786186_0001_1_01 [E48A74DAC465470382CB4BF3D9B3D025]
2016-01-14 17:16:26,573 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:26,573 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791786186_0001_1][Event:VERTEX_FINISHED]: vertexName=E48A74DAC465470382CB4BF3D9B3D025, vertexId=vertex_1452791786186_0001_1_01, initRequestedTime=1452791786374, initedTime=1452791786389, startRequestedTime=1452791786403, startedTime=1452791786403, finishTime=1452791786572, timeTaken=169, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=187, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791786560, Process_Duration=6, Process_End_Time=1452791786566, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452791786519, firstTasksToStart=[ task_1452791786186_0001_1_01_000000 ], lastTaskFinishTime=1452791786569, lastTasksToFinish=[ task_1452791786186_0001_1_01_000000 ], minTaskDuration=50, maxTaskDuration=50, avgTaskDuration=50.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791786186_0001_1_01_000000 ], longestDurationTasks=[ task_1452791786186_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:26,574 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791786186_0001_1_01 [E48A74DAC465470382CB4BF3D9B3D025] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:26,574 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791786186_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:26,575 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791786186_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791786186_0001_00_000002, stoppedTime=1452791786575, exitStatus=0
2016-01-14 17:16:26,575 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791786186_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:26,575 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791786186_0001_1_01 [E48A74DAC465470382CB4BF3D9B3D025] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:26,575 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791786186_0001_1
2016-01-14 17:16:26,575 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:26,575 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791786186_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452791786186_0001_1
2016-01-14 17:16:26,576 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 27FF6E394A7248CD83546677F4B0BBDD for vertex: vertex_1452791786186_0001_1_01 [E48A74DAC465470382CB4BF3D9B3D025]
2016-01-14 17:16:26,577 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=187, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791786560, Process_Duration=6, Process_End_Time=1452791786566, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 17:16:26,577 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:26,578 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:26,578 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791786186_0001_00_000002
2016-01-14 17:16:26,578 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791786186_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 17:16:26,579 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791786186_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:26,579 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791786186_0001_00_000002, Exiting
2016-01-14 17:16:26,579 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791786186_0001_00_000002
2016-01-14 17:16:26,579 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791786186_0001_00_000002
2016-01-14 17:16:26,580 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791786186_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:26,580 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791786186_0001_00_000002 completed successfully
2016-01-14 17:16:26,585 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452791786186_0001_1_00 [F2542EDAE8564AC6BC9AD993FBE8C96B]
2016-01-14 17:16:26,587 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791786186_0001_1, queueSize=0
2016-01-14 17:16:26,587 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:26,588 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791786186_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791786186_0001_1, startTime=1452791786372, finishTime=1452791786585, timeTaken=213, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=520, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1028653056, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905583573046, Process_Duration=9, Process_End_Time=2905583573055, Read_Duration=1, Tuples_Read=40, Tuples_Written=40, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 17:16:26,589 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791786186_0001_1 finished with state: SUCCEEDED
2016-01-14 17:16:26,589 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791786186_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:26,589 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791786186_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:16:26 Completed Dag: dag_1452791786186_0001_1
2016-01-14 17:16:26,589 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452791786186_0001_00_000002
2016-01-14 17:16:26,589 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791786186_0001_1, dagState=SUCCEEDED
2016-01-14 17:16:26,590 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791786186_0001_1_post
2016-01-14 17:16:26,590 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:26,590 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:26,597 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...yplatform/-1_UNKNOWN_SWAP, applicationId=application_1452791786186_0001
2016-01-14 17:16:26,600 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:26,600 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:26,600 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:26,600 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:26,600 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:26,601 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:26,602 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:26,602 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:26,602 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:26,602 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:26,602 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:26,603 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:26,604 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:26,607 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/399A2353C8D447EFAE57D50D3ED33D46/CBFB840D95BE4B8E9769463E4B40B4C7/.tez/application_1452791786186_0001
2016-01-14 17:16:26,608 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.442
2016-01-14 17:16:26,683 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_UNKNOWN_SWAP
2016-01-14 17:16:26,706 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:26,706 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:26,740 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-01-14 17:16:26,751 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.021
2016-01-14 17:16:26,752 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:26,752 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:26,752 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:26,767 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:26,773 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_-1
2016-01-14 17:16:26,818 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:26,819 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:26,819 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_-1"]
2016-01-14 17:16:26,819 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:26,819 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:26,820 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:26,820 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...mblyplatform/-1_VALUES_-1
2016-01-14 17:16:26,821 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:26,834 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:26,839 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:26,839 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:26,840 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:26,849 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5640A02C36FA486E8DF94A06C1C1F338/B74121D25A014A2E811DA8EC65C139A7/.tez/application_1452791786839_0001 doesn't exist and is created
2016-01-14 17:16:26,870 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:26,870 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:26,870 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5640A02C36FA486E8DF94A06C1C1F338/B74121D25A014A2E811DA8EC65C139A7/.tez/application_1452791786839_0001
2016-01-14 17:16:26,875 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791786839_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:26,886 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:26,887 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:26,889 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:26,890 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:26,890 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5640A02C36FA486E8DF94A06C1C1F338/B74121D25A014A2E811DA8EC65C139A7/.tez/application_1452791786839_0001/localmode-log-dir
2016-01-14 17:16:26,891 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5640A02C36FA486E8DF94A06C1C1F338/B74121D25A014A2E811DA8EC65C139A7/.tez/application_1452791786839_0001/localmode-log-dir/history.txt.appattempt_1452791786839_0001_000000, maxErrors=10
2016-01-14 17:16:26,891 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:26,892 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791786839_0001_000000, appSubmitTime=1452791786875, launchTime=1452791786875
2016-01-14 17:16:26,896 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:26,897 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:26,899 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:26,900 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:26,901 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:42801
2016-01-14 17:16:26,903 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:26,907 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791786839_0001_000000, startTime=1452791786907
2016-01-14 17:16:26,907 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:26,907 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:26,970 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:26,970 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:26,970 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...mblyplatform/-1_VALUES_-1, applicationId=application_1452791786839_0001, dagName=[5640A02C36FA486E8DF94A06C1C1F338/B74121D25A014A2E811DA8EC65C139A7] every/(1/1) ...mblyplatform/-1_VALUES_-1
2016-01-14 17:16:26,994 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [5640A02C36FA486E8DF94A06C1C1F338/B74121D25A014A2E811DA8EC65C139A7] every/(1/1) ...mblyplatform/-1_VALUES_-1
2016-01-14 17:16:26,997 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791786839_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5640A02C36FA486E8DF94A06C1C1F338/B74121D25A014A2E811DA8EC65C139A7/.tez/application_1452791786839_0001/localmode-log-dir/dag_1452791786839_0001_1.dot
2016-01-14 17:16:26,997 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5640A02C36FA486E8DF94A06C1C1F338/B74121D25A014A2E811DA8EC65C139A7/.tez/application_1452791786839_0001/localmode-log-dir/dag_1452791786839_0001_1-tez-dag.pb.txt
2016-01-14 17:16:27,007 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791786839_0001_1
2016-01-14 17:16:27,007 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:27,007 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [5640A02C36FA486E8DF94A06C1C1F338/B74121D25A014A2E811DA8EC65C139A7] every/(1/1) ...mblyplatform/-1_VALUES_-1
2016-01-14 17:16:27 Running Dag: dag_1452791786839_0001_1
2016-01-14 17:16:27,021 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:27,022 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791786839_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791786839_0001_1, submitTime=1452791786995
2016-01-14 17:16:27,023 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex C86C11B10BF443AC96804CCB78FC742A
2016-01-14 17:16:27,023 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:27,023 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:27,024 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791786839_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791786839_0001_1, initTime=1452791787022
2016-01-14 17:16:27,024 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791786839_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:27,026 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791786839_0001_1][Event:DAG_STARTED]: dagID=dag_1452791786839_0001_1, startTime=1452791787024
2016-01-14 17:16:27,026 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:27,026 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791786839_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:27,026 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: BA7C541BB1E941788445F27ACE3320D4 : {60068D588B644D689658DB1D3597CC60={InputName=60068D588B644D689658DB1D3597CC60}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:27,027 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 60068D588B644D689658DB1D3597CC60, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:27,027 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791786839_0001_1_00 [BA7C541BB1E941788445F27ACE3320D4]
2016-01-14 17:16:27,027 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791786839_0001_1_00 [BA7C541BB1E941788445F27ACE3320D4]
2016-01-14 17:16:27,027 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791786839_0001_1_00 [BA7C541BB1E941788445F27ACE3320D4]
2016-01-14 17:16:27,027 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791786839_0001_1_00 [BA7C541BB1E941788445F27ACE3320D4]. Starting root input initializers: 1
2016-01-14 17:16:27,026 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:27,028 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791786839_0001_1_00 [BA7C541BB1E941788445F27ACE3320D4] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:27,028 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791786839_0001_1_01 [C86C11B10BF443AC96804CCB78FC742A]
2016-01-14 17:16:27,027 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...mblyplatform/-1_VALUES_-1, applicationId=application_1452791786839_0001, dagName=[5640A02C36FA486E8DF94A06C1C1F338/B74121D25A014A2E811DA8EC65C139A7] every/(1/1) ...mblyplatform/-1_VALUES_-1
2016-01-14 17:16:27,031 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:27,032 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791786839_0001_1_01 [C86C11B10BF443AC96804CCB78FC742A]
2016-01-14 17:16:27,032 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791786839_0001_1_01 [C86C11B10BF443AC96804CCB78FC742A]
2016-01-14 17:16:27,032 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791786839_0001_1_01 [C86C11B10BF443AC96804CCB78FC742A]
2016-01-14 17:16:27,032 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=B02D599C3FA7449E8E476AECF7C5ACBF, vertexId=vertex_1452791786839_0001_1_01 [C86C11B10BF443AC96804CCB78FC742A], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:27,032 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=B02D599C3FA7449E8E476AECF7C5ACBF, vertexId=vertex_1452791786839_0001_1_01 [C86C11B10BF443AC96804CCB78FC742A]
2016-01-14 17:16:27,032 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 60068D588B644D689658DB1D3597CC60 on vertex vertex_1452791786839_0001_1_00 [BA7C541BB1E941788445F27ACE3320D4]
2016-01-14 17:16:27,034 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=B02D599C3FA7449E8E476AECF7C5ACBF, vertexName=C86C11B10BF443AC96804CCB78FC742A, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:27,034 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=B02D599C3FA7449E8E476AECF7C5ACBF, vertexId=vertex_1452791786839_0001_1_01 [C86C11B10BF443AC96804CCB78FC742A]
2016-01-14 17:16:27,035 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:27,035 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 60068D588B644D689658DB1D3597CC60 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:27,036 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:27,036 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:27,036 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791786839_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C86C11B10BF443AC96804CCB78FC742A, vertexId=vertex_1452791786839_0001_1_01, initRequestedTime=1452791787028, initedTime=1452791787036, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:27,036 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791786839_0001_1_01 [C86C11B10BF443AC96804CCB78FC742A] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:27,036 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:27,038 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:27,038 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 60068D588B644D689658DB1D3597CC60 on vertex vertex_1452791786839_0001_1_00 [BA7C541BB1E941788445F27ACE3320D4]
2016-01-14 17:16:27,039 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791786839_0001_1_00 [BA7C541BB1E941788445F27ACE3320D4] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:27,039 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {60068D588B644D689658DB1D3597CC60=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:27,039 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791786839_0001_1_00 [BA7C541BB1E941788445F27ACE3320D4] parallelism set to 1
2016-01-14 17:16:27,040 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791786839_0001_1_00 [BA7C541BB1E941788445F27ACE3320D4] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:27,040 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791786839_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791786839_0001_1_00, eventCount=1
2016-01-14 17:16:27,041 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:27,041 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791786839_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BA7C541BB1E941788445F27ACE3320D4, vertexId=vertex_1452791786839_0001_1_00, initRequestedTime=1452791787026, initedTime=1452791787040, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:27,042 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791786839_0001_1_00 [BA7C541BB1E941788445F27ACE3320D4] with distanceFromRoot: 0
2016-01-14 17:16:27,042 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791786839_0001_1_00 [BA7C541BB1E941788445F27ACE3320D4] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:27,042 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in BA7C541BB1E941788445F27ACE3320D4
2016-01-14 17:16:27,042 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791786839_0001_1_00 [BA7C541BB1E941788445F27ACE3320D4]
2016-01-14 17:16:27,043 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:27,043 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791786839_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791786839_0001_1_00, startRequestedTime=1452791787042, startedTime=1452791787042
2016-01-14 17:16:27,044 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791786839_0001_1_00 [BA7C541BB1E941788445F27ACE3320D4] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:27,044 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:27,045 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791786839_0001_1][Event:TASK_STARTED]: vertexName=BA7C541BB1E941788445F27ACE3320D4, taskId=task_1452791786839_0001_1_00_000000, scheduledTime=1452791787044, launchTime=1452791787044
2016-01-14 17:16:27,045 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791786839_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:27,045 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791786839_0001_1_00 for vertex: vertex_1452791786839_0001_1_01 [C86C11B10BF443AC96804CCB78FC742A] numStartedSources: 1 numSources: 1
2016-01-14 17:16:27,045 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791786839_0001_1_01 [C86C11B10BF443AC96804CCB78FC742A] with distanceFromRoot: 1
2016-01-14 17:16:27,046 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791786839_0001_1_00 [BA7C541BB1E941788445F27ACE3320D4]
2016-01-14 17:16:27,046 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: BA7C541BB1E941788445F27ACE3320D4 in vertex: C86C11B10BF443AC96804CCB78FC742A
2016-01-14 17:16:27,046 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: C86C11B10BF443AC96804CCB78FC742A with 1 source tasks and 1 pending tasks
2016-01-14 17:16:27,047 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:27,047 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791786839_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791786839_0001_1_01, startRequestedTime=1452791787046, startedTime=1452791787046
2016-01-14 17:16:27,047 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791786839_0001_1_01 [C86C11B10BF443AC96804CCB78FC742A] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:27,048 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [5640A02C36FA486E8DF94A06C1C1F338/B74121D25A014A2E811DA8EC65C139A7] every/(1/1) ...mblyplatform/-1_VALUES_-1, VertexName: BA7C541BB1E941788445F27ACE3320D4, VertexParallelism: 1, TaskAttemptID:attempt_1452791786839_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=60068D588B644D689658DB1D3597CC60, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C86C11B10BF443AC96804CCB78FC742A, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:27,048 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791786839_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:27,048 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:27,049 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:27,049 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:27,050 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:27,050 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791786839_0001_00_000001
2016-01-14 17:16:27,051 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791786839_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:27,052 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791786839_0001_00_000001
2016-01-14 17:16:27,052 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:27,053 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791786839_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:27,053 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:27,054 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791786839_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791786839_0001_00_000001, launchTime=1452791787053
2016-01-14 17:16:27,054 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791786839_0001_00_000001
2016-01-14 17:16:27,055 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791786839_0001_1_00_000000_0] to container: [container_1452791786839_0001_00_000001]
2016-01-14 17:16:27,055 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791786839_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:27,056 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791786839_0001_1_00_000000_0] started. Is using containerId: [container_1452791786839_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:27,056 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791786839_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BA7C541BB1E941788445F27ACE3320D4, taskAttemptId=attempt_1452791786839_0001_1_00_000000_0, startTime=1452791787056, containerId=container_1452791786839_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791786839_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:27,056 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791786839_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:27,057 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791786839_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:27,057 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:27,057 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791786839_0001, with dag id: dag_1452791786839_0001_1
2016-01-14 17:16:27,057 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791786839_0001_00_000001 given task: attempt_1452791786839_0001_1_00_000000_0
2016-01-14 17:16:27,058 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791786839_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791786839_0001_1_00_000000_0
2016-01-14 17:16:27 Starting to run new task attempt: attempt_1452791786839_0001_1_00_000000_0
2016-01-14 17:16:27,066 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791786839_0001_1_00_000000_0
2016-01-14 17:16:27,066 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:27,066 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:27,066 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:27,067 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:27,067 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:27,067 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:27,067 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [5640A02C36FA486E8DF94A06C1C1F338/B74121D25A014A2E811DA8EC65C139A7] every/(1/1) ...mblyplatform/-1_VALUES_-1, VertexName: BA7C541BB1E941788445F27ACE3320D4, VertexParallelism: 1, TaskAttemptID:attempt_1452791786839_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=60068D588B644D689658DB1D3597CC60, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C86C11B10BF443AC96804CCB78FC742A, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:27,067 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:16:27,070 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791786839_0001_1_00_000000_0
2016-01-14 17:16:27,070 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:27,070 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:27,071 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:27,071 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791786839_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:27,075 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:27,080 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=C86C11B10BF443AC96804CCB78FC742A, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:27,082 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=60068D588B644D689658DB1D3597CC60, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:27,084 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:27,084 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 60068D588B644D689658DB1D3597CC60
2016-01-14 17:16:27,084 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 60068D588B644D689658DB1D3597CC60
2016-01-14 17:16:27,084 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:27,085 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: C86C11B10BF443AC96804CCB78FC742A
2016-01-14 17:16:27,086 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:27,087 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 60068D588B644D689658DB1D3597CC60
2016-01-14 17:16:27,087 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 60068D588B644D689658DB1D3597CC60
2016-01-14 17:16:27,089 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:27,090 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: C86C11B10BF443AC96804CCB78FC742A
2016-01-14 17:16:27,090 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BA7C541BB1E941788445F27ACE3320D4, ordinal: 0
2016-01-14 17:16:27,091 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BA7C541BB1E941788445F27ACE3320D4, mem on start (mb), free: 54, total: 490, max: 672
2016-01-14 17:16:27,091 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:27,091 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:27,092 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:27,091 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: C86C11B10BF443AC96804CCB78FC742A
2016-01-14 17:16:27,092 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:27,092 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:27,093 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:27,093 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:27,093 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:27,094 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:27,094 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 17:16:27,094 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 60068D588B644D689658DB1D3597CC60, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:27,095 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, C86C11B10BF443AC96804CCB78FC742A, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:27,095 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:27,095 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 60068D588B644D689658DB1D3597CC60 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:27,095 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 60068D588B644D689658DB1D3597CC60
2016-01-14 17:16:27,096 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:27,096 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 60068D588B644D689658DB1D3597CC60
2016-01-14 17:16:27,095 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:27,096 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:27,096 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:27,097 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791786839_0001_1_00_000000_0
2016-01-14 17:16:27,099 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:27,100 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 60068D588B644D689658DB1D3597CC60
2016-01-14 17:16:27,100 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 3842D65BC97544FDA1500DF09F079BE8
2016-01-14 17:16:27,100 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 3842D65BC97544FDA1500DF09F079BE8
2016-01-14 17:16:27,100 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:27,101 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@23b0000d; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@77d367c8; comparator=cascading.tuple.hadoop.util.TupleComparator@5842b8ed; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:27,101 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@47afce0
2016-01-14 17:16:27,101 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:27,103 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:27,103 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:27,103 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@47afce0
2016-01-14 17:16:27,323 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:27,324 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:27,336 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:27,338 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:27,338 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:27,338 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:27,339 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:27,341 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:27,341 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:27,341 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BA7C541BB1E941788445F27ACE3320D4, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:27,344 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791786839_0001_1_00_000000_0
2016-01-14 17:16:27,344 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BA7C541BB1E941788445F27ACE3320D4, mem on close (mb), free: 249, total: 372, max: 672
2016-01-14 17:16:27,344 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:27,344 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:27,344 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:27,345 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:27,348 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:27 Completed running task attempt: attempt_1452791786839_0001_1_00_000000_0
2016-01-14 17:16:27,353 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791786839_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:27,354 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791786839_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791786839_0001_1_00, eventCount=1
2016-01-14 17:16:27,354 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791786839_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BA7C541BB1E941788445F27ACE3320D4, taskAttemptId=attempt_1452791786839_0001_1_00_000000_0, startTime=1452791787056, finishTime=1452791787354, timeTaken=298, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=219, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=390594560, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791787341, Process_Duration=3, Process_End_Time=1452791787344, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:27,354 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791786839_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:27,354 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=219, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=390594560, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791787341, Process_Duration=3, Process_End_Time=1452791787344, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:27,354 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791786839_0001_1_00_000000_0
2016-01-14 17:16:27,355 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:27,355 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:27,354 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:27,354 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:27,356 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791786839_0001_00_000001
2016-01-14 17:16:27,357 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:27,356 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791786839_0001_1][Event:TASK_FINISHED]: vertexName=BA7C541BB1E941788445F27ACE3320D4, taskId=task_1452791786839_0001_1_00_000000, startTime=1452791787056, finishTime=1452791787355, timeTaken=299, status=SUCCEEDED, successfulAttemptID=attempt_1452791786839_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=219, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=390594560, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791787341, Process_Duration=3, Process_End_Time=1452791787344, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:27,357 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791786839_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:27,357 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791786839_0001_1_00 [BA7C541BB1E941788445F27ACE3320D4] : 1
2016-01-14 17:16:27,358 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791786839_0001_1_00 [BA7C541BB1E941788445F27ACE3320D4]
2016-01-14 17:16:27,358 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:27,359 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791786839_0001_1][Event:VERTEX_FINISHED]: vertexName=BA7C541BB1E941788445F27ACE3320D4, vertexId=vertex_1452791786839_0001_1_00, initRequestedTime=1452791787026, initedTime=1452791787040, startRequestedTime=1452791787042, startedTime=1452791787042, finishTime=1452791787358, timeTaken=316, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=219, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=390594560, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791787341, Process_Duration=3, Process_End_Time=1452791787344, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791787056, firstTasksToStart=[ task_1452791786839_0001_1_00_000000 ], lastTaskFinishTime=1452791787354, lastTasksToFinish=[ task_1452791786839_0001_1_00_000000 ], minTaskDuration=298, maxTaskDuration=298, avgTaskDuration=298.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791786839_0001_1_00_000000 ], longestDurationTasks=[ task_1452791786839_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:27,359 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791786839_0001_1_00 [BA7C541BB1E941788445F27ACE3320D4] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:27,359 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791786839_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:27,359 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791786839_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791786839_0001_00_000001, stoppedTime=1452791787359, exitStatus=0
2016-01-14 17:16:27,359 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:27,359 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791786839_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:27,360 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791786839_0001_1_01 [C86C11B10BF443AC96804CCB78FC742A] attempt: attempt_1452791786839_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:27,360 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: C86C11B10BF443AC96804CCB78FC742A
2016-01-14 17:16:27,360 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791786839_0001_1_01 [C86C11B10BF443AC96804CCB78FC742A]
2016-01-14 17:16:27,360 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791786839_0001_1_00 [BA7C541BB1E941788445F27ACE3320D4] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:27,360 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791786839_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:27,362 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791786839_0001_1][Event:TASK_STARTED]: vertexName=C86C11B10BF443AC96804CCB78FC742A, taskId=task_1452791786839_0001_1_01_000000, scheduledTime=1452791787362, launchTime=1452791787362
2016-01-14 17:16:27,362 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791786839_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:27,362 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791786839_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:27,362 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791786839_0001_1_01 [C86C11B10BF443AC96804CCB78FC742A]
2016-01-14 17:16:27,362 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:27,362 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [5640A02C36FA486E8DF94A06C1C1F338/B74121D25A014A2E811DA8EC65C139A7] every/(1/1) ...mblyplatform/-1_VALUES_-1, VertexName: C86C11B10BF443AC96804CCB78FC742A, VertexParallelism: 1, TaskAttemptID:attempt_1452791786839_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BA7C541BB1E941788445F27ACE3320D4, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=B02D599C3FA7449E8E476AECF7C5ACBF, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:27,363 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791786839_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:27,363 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:27,364 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791786839_0001_00_000002
2016-01-14 17:16:27,364 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791786839_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:27,364 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 17:16:27,364 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 17:16:27,365 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791786839_0001_00_000002
2016-01-14 17:16:27,365 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:27,366 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791786839_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:27,366 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:27,366 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791786839_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791786839_0001_00_000002, launchTime=1452791787366
2016-01-14 17:16:27,367 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791786839_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:27,367 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791786839_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:27,367 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791786839_0001_00_000001, Exiting
2016-01-14 17:16:27,367 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791786839_0001_00_000001
2016-01-14 17:16:27,367 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791786839_0001_00_000001
2016-01-14 17:16:27,367 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791786839_0001_00_000002
2016-01-14 17:16:27,369 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791786839_0001_1_01_000000_0] to container: [container_1452791786839_0001_00_000002]
2016-01-14 17:16:27,369 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791786839_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:27,369 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791786839_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:27,369 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791786839_0001_00_000001 completed successfully
2016-01-14 17:16:27,370 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791786839_0001_1_01_000000_0] started. Is using containerId: [container_1452791786839_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:27,370 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791786839_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C86C11B10BF443AC96804CCB78FC742A, taskAttemptId=attempt_1452791786839_0001_1_01_000000_0, startTime=1452791787369, containerId=container_1452791786839_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791786839_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:27,370 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791786839_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:27,370 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:27,370 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791786839_0001_00_000002 given task: attempt_1452791786839_0001_1_01_000000_0
2016-01-14 17:16:27,370 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791786839_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:27,371 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791786839_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:27,371 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791786839_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791786839_0001_1_01_000000_0
2016-01-14 17:16:27 Starting to run new task attempt: attempt_1452791786839_0001_1_01_000000_0
2016-01-14 17:16:27,372 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791786839_0001_1_01_000000_0
2016-01-14 17:16:27,372 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:27,372 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:27,372 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:27,373 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:27,373 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:27,373 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:27,373 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [5640A02C36FA486E8DF94A06C1C1F338/B74121D25A014A2E811DA8EC65C139A7] every/(1/1) ...mblyplatform/-1_VALUES_-1, VertexName: C86C11B10BF443AC96804CCB78FC742A, VertexParallelism: 1, TaskAttemptID:attempt_1452791786839_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BA7C541BB1E941788445F27ACE3320D4, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=B02D599C3FA7449E8E476AECF7C5ACBF, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:27,373 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:16:27,374 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791786839_0001_1_01_000000_0
2016-01-14 17:16:27,374 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:27,374 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:27,375 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:27,375 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=BA7C541BB1E941788445F27ACE3320D4, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:27,375 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=B02D599C3FA7449E8E476AECF7C5ACBF, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:27,375 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:27,376 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: BA7C541BB1E941788445F27ACE3320D4
2016-01-14 17:16:27,376 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:27,377 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: B02D599C3FA7449E8E476AECF7C5ACBF
2016-01-14 17:16:27,377 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:27,377 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: B02D599C3FA7449E8E476AECF7C5ACBF
2016-01-14 17:16:27,378 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:27,380 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:27,381 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:27,381 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: BA7C541BB1E941788445F27ACE3320D4
2016-01-14 17:16:27,384 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: BA7C541BB1E941788445F27ACE3320D4
2016-01-14 17:16:27,384 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791786839_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:27,385 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C86C11B10BF443AC96804CCB78FC742A, ordinal: 1
2016-01-14 17:16:27,385 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C86C11B10BF443AC96804CCB78FC742A, mem on start (mb), free: 243, total: 372, max: 672
2016-01-14 17:16:27,385 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:27,385 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:27,385 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:27,391 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:27,391 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: B02D599C3FA7449E8E476AECF7C5ACBF
2016-01-14 17:16:27,391 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:27,392 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:27,392 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:27,392 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:27,392 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:27,392 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:27,392 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:27,393 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, B02D599C3FA7449E8E476AECF7C5ACBF, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:27,393 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, BA7C541BB1E941788445F27ACE3320D4, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 17:16:27,393 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:27,393 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: BA7C541BB1E941788445F27ACE3320D4 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:27,393 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: BA7C541BB1E941788445F27ACE3320D4
2016-01-14 17:16:27,393 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:27,393 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:27,395 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:27,395 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:27,396 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: BA7C541BB1E941788445F27ACE3320D4 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:27,396 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 17:16:27,396 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:27,396 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:27,397 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: BA7C541BB1E941788445F27ACE3320D4
2016-01-14 17:16:27,397 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:27,398 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791786839_0001_1_01_000000_0
2016-01-14 17:16:27,399 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791786839_0001_1_00_000000_0_10062, runDuration: 264726, ]
2016-01-14 17:16:27,400 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:27,400 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791786839_0001_1_00_000000_0_10062] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791787400, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 17:16:27,400 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : BA7C541BB1E941788445F27ACE3320D4
2016-01-14 17:16:27,401 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:27,401 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [BA7C541BB1E941788445F27ACE3320D4] #11 in 2ms
2016-01-14 17:16:27,401 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 3842D65BC97544FDA1500DF09F079BE8
2016-01-14 17:16:27,401 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_-1"], id: B02D599C3FA7449E8E476AECF7C5ACBF
2016-01-14 17:16:27,401 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_-1"]
2016-01-14 17:16:27,402 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 3842D65BC97544FDA1500DF09F079BE8, for 1 inputs
2016-01-14 17:16:27,402 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [BA7C541BB1E941788445F27ACE3320D4] #11, status:false, isInterrupted:false
2016-01-14 17:16:27,402 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:27,402 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:27,403 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:27,403 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:27,403 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:27,403 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : BA7C541BB1E941788445F27ACE3320D4
2016-01-14 17:16:27,403 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:27,403 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C86C11B10BF443AC96804CCB78FC742A, all 1 inputs ready in: 00:00:00.001
2016-01-14 17:16:27,404 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@1ed9f356; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@76f6a2ec
2016-01-14 17:16:27,405 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:27,412 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(92)) - caught throwable
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:27,414 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:27,414 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:27,415 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791786839_0001_1_01_000000_0
2016-01-14 17:16:27,415 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791786839_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:27,416 INFO  task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1452791786839_0001_1_01_000000_0
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:27,420 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=390594560, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791787403, Process_Duration=13, Process_End_Time=1452791787416, Read_Duration=0, Tuples_Read=1]]
2016-01-14 17:16:27,421 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:27,420 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791786839_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C86C11B10BF443AC96804CCB78FC742A, taskAttemptId=attempt_1452791786839_0001_1_01_000000_0, startTime=1452791787369, finishTime=1452791787420, timeTaken=51, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=390594560, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791787403, Process_Duration=13, Process_End_Time=1452791787416, Read_Duration=0, Tuples_Read=1
2016-01-14 17:16:27,422 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791786839_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-01-14 17:16:27,423 INFO  impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1452791786839_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-01-14 17:16:27,423 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791786839_0001_1][Event:TASK_FINISHED]: vertexName=C86C11B10BF443AC96804CCB78FC742A, taskId=task_1452791786839_0001_1_01_000000, startTime=1452791787369, finishTime=1452791787423, timeTaken=54, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
], counters=Counters: 0
2016-01-14 17:16:27,423 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:27,423 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791786839_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-01-14 17:16:27,425 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791786839_0001_1_01 [C86C11B10BF443AC96804CCB78FC742A] : 1
2016-01-14 17:16:27,425 INFO  impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1452791786839_0001_1_01 [C86C11B10BF443AC96804CCB78FC742A] because task failed: task_1452791786839_0001_1_01_000000
2016-01-14 17:16:27,425 INFO  impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1452791786839_0001_1_01 [C86C11B10BF443AC96804CCB78FC742A] due to trigger: OWN_TASK_FAILURE
2016-01-14 17:16:27,425 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-01-14 17:16:27,425 INFO  impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1452791786839_0001_1_01 [C86C11B10BF443AC96804CCB78FC742A]
2016-01-14 17:16:27,426 INFO  impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=B02D599C3FA7449E8E476AECF7C5ACBF, vertexId=vertex_1452791786839_0001_1_01 [C86C11B10BF443AC96804CCB78FC742A]
2016-01-14 17:16:27,427 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C86C11B10BF443AC96804CCB78FC742A, mem on close (mb), free: 239, total: 372, max: 672
2016-01-14 17:16:27,427 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=C86C11B10BF443AC96804CCB78FC742A, index=1
2016-01-14 17:16:27,427 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:27,427 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:27,427 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791786839_0001_1][Event:VERTEX_FINISHED]: vertexName=C86C11B10BF443AC96804CCB78FC742A, vertexId=vertex_1452791786839_0001_1_01, initRequestedTime=1452791787028, initedTime=1452791787036, startRequestedTime=1452791787046, startedTime=1452791787046, finishTime=1452791787427, timeTaken=381, status=FAILED, diagnostics=Task failed, taskId=task_1452791786839_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1452791786839_0001_1_01 [C86C11B10BF443AC96804CCB78FC742A] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1452791787369, firstTasksToStart=[ task_1452791786839_0001_1_01_000000 ], lastTaskFinishTime=1452791787420, lastTasksToFinish=[ task_1452791786839_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[  ], longestDurationTasks=[  ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-01-14 17:16:27,428 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791786839_0001_1_01 [C86C11B10BF443AC96804CCB78FC742A] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-01-14 17:16:27,428 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791786839_0001_1_01 [C86C11B10BF443AC96804CCB78FC742A] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-01-14 17:16:27,428 INFO  impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-01-14 17:16:27,428 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791786839_0001_1
2016-01-14 17:16:27,429 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1452791786839_0001_1_00 [BA7C541BB1E941788445F27ACE3320D4]
2016-01-14 17:16:27,429 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: B02D599C3FA7449E8E476AECF7C5ACBF for vertex: vertex_1452791786839_0001_1_01 [C86C11B10BF443AC96804CCB78FC742A]
2016-01-14 17:16:27,429 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: BA7C541BB1E941788445F27ACE3320D4
2016-01-14 17:16:27,429 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:27,430 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:27,430 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:27,430 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:27,430 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:27,430 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=C86C11B10BF443AC96804CCB78FC742A, sourceVertex=BA7C541BB1E941788445F27ACE3320D4
2016-01-14 17:16:27,430 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=C86C11B10BF443AC96804CCB78FC742A, sourceVertex=B02D599C3FA7449E8E476AECF7C5ACBF
2016-01-14 17:16:27,431 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791786839_0001_00_000002
2016-01-14 17:16:27,431 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791786839_0001_00_000002
2016-01-14 17:16:27,431 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791786839_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-01-14 17:16:27,431 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1452791786839_0001_00_000002 completed but with errors
2016-01-14 17:16:27,431 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791786839_0001_1, queueSize=0
2016-01-14 17:16:27,432 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:27,434 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791786839_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791786839_0001_1, startTime=1452791787024, finishTime=1452791787429, timeTaken=405, status=FAILED, diagnostics=Vertex failed, vertexName=C86C11B10BF443AC96804CCB78FC742A, vertexId=vertex_1452791786839_0001_1_01, diagnostics=[Task failed, taskId=task_1452791786839_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1452791786839_0001_1_01 [C86C11B10BF443AC96804CCB78FC742A] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=219, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=390594560, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791787341, Process_Duration=3, Process_End_Time=1452791787344, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:27,434 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791786839_0001_1 finished with state: FAILED
2016-01-14 17:16:27,434 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791786839_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:27,435 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:27,435 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791786839_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791786839_0001_00_000002, stoppedTime=1452791787434, exitStatus=0
2016-01-14 17:16:27,435 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791786839_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:27,435 INFO  node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1452791786839_0001_1_01_000000_0 failed: true container: container_1452791786839_0001_00_000002 numFailedTAs: 0
2016-01-14 17:16:27,436 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1452791786839_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-01-14 17:16:27,436 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791786839_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:16:27 Completed Dag: dag_1452791786839_0001_1
2016-01-14 17:16:27,437 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791786839_0001_1, dagState=FAILED
2016-01-14 17:16:27,437 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791786839_0001_1_post
2016-01-14 17:16:27,437 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:27,438 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:27,438 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452791786839_0001_00_000002
2016-01-14 17:16:27,439 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791786839_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-01-14 17:16:27,446 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...mblyplatform/-1_VALUES_-1, applicationId=application_1452791786839_0001
2016-01-14 17:16:27,451 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:27,451 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:27,451 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:27,451 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:27,451 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:27,452 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:27,454 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:27,454 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:27,455 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:27,455 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:27,455 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:27,455 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:27,455 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:27,462 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5640A02C36FA486E8DF94A06C1C1F338/B74121D25A014A2E811DA8EC65C139A7/.tez/application_1452791786839_0001
2016-01-14 17:16:27,463 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping all jobs
2016-01-14 17:16:27,463 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping: (1/1) ...mblyplatform/-1_VALUES_-1
2016-01-14 17:16:27,463 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopped all jobs
2016-01-14 17:16:27,464 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.646
2016-01-14 17:16:27,464 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_VALUES_-1
2016-01-14 17:16:27,488 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:27,488 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:27,518 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.006
2016-01-14 17:16:27,518 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.006
2016-01-14 17:16:27,519 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 17:16:27,520 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 17:16:27,520 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_VALUES_'field'
2016-01-14 17:16:27,542 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:27,542 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:27,583 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.016
2016-01-14 17:16:27,589 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.022
2016-01-14 17:16:27,590 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:27,590 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:27,590 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:27,605 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:27,611 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_RESULTS
2016-01-14 17:16:27,656 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:27,656 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:27,657 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_RESULTS"]
2016-01-14 17:16:27,657 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:27,657 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:27,658 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:27,659 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...latform/-1_VALUES_RESULTS
2016-01-14 17:16:27,659 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:27,671 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:27,676 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:27,676 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:27,676 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:27,685 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/739CF7EDADD34C428AED87D51F4EE678/A229578C3EC840FD93BF90F84E0FBCE6/.tez/application_1452791787676_0001 doesn't exist and is created
2016-01-14 17:16:27,702 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:27,702 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:27,702 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/739CF7EDADD34C428AED87D51F4EE678/A229578C3EC840FD93BF90F84E0FBCE6/.tez/application_1452791787676_0001
2016-01-14 17:16:27,705 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791787676_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:27,715 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:27,716 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:27,718 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:27,718 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:27,718 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/739CF7EDADD34C428AED87D51F4EE678/A229578C3EC840FD93BF90F84E0FBCE6/.tez/application_1452791787676_0001/localmode-log-dir
2016-01-14 17:16:27,718 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/739CF7EDADD34C428AED87D51F4EE678/A229578C3EC840FD93BF90F84E0FBCE6/.tez/application_1452791787676_0001/localmode-log-dir/history.txt.appattempt_1452791787676_0001_000000, maxErrors=10
2016-01-14 17:16:27,718 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:27,719 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791787676_0001_000000, appSubmitTime=1452791787705, launchTime=1452791787705
2016-01-14 17:16:27,724 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:27,726 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:40094
2016-01-14 17:16:27,727 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:27,729 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:27,729 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:27,730 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:27,732 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:27,732 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791787676_0001_000000, startTime=1452791787732
2016-01-14 17:16:27,733 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:27,802 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:27,803 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:27,803 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/-1_VALUES_RESULTS, applicationId=application_1452791787676_0001, dagName=[739CF7EDADD34C428AED87D51F4EE678/A229578C3EC840FD93BF90F84E0FBCE6] every/(1/1) ...latform/-1_VALUES_RESULTS
2016-01-14 17:16:27,818 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [739CF7EDADD34C428AED87D51F4EE678/A229578C3EC840FD93BF90F84E0FBCE6] every/(1/1) ...latform/-1_VALUES_RESULTS
2016-01-14 17:16:27,820 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791787676_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/739CF7EDADD34C428AED87D51F4EE678/A229578C3EC840FD93BF90F84E0FBCE6/.tez/application_1452791787676_0001/localmode-log-dir/dag_1452791787676_0001_1.dot
2016-01-14 17:16:27,820 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/739CF7EDADD34C428AED87D51F4EE678/A229578C3EC840FD93BF90F84E0FBCE6/.tez/application_1452791787676_0001/localmode-log-dir/dag_1452791787676_0001_1-tez-dag.pb.txt
2016-01-14 17:16:27,827 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791787676_0001_1
2016-01-14 17:16:27,827 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:27,827 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [739CF7EDADD34C428AED87D51F4EE678/A229578C3EC840FD93BF90F84E0FBCE6] every/(1/1) ...latform/-1_VALUES_RESULTS
2016-01-14 17:16:27 Running Dag: dag_1452791787676_0001_1
2016-01-14 17:16:27,840 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791787676_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791787676_0001_1, submitTime=1452791787818
2016-01-14 17:16:27,840 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:27,841 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex B2E7CCFF712E46AFB3E8D496A4812B19
2016-01-14 17:16:27,841 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:27,841 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791787676_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791787676_0001_1, initTime=1452791787840
2016-01-14 17:16:27,841 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:27,842 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791787676_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:27,842 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791787676_0001_1][Event:DAG_STARTED]: dagID=dag_1452791787676_0001_1, startTime=1452791787842
2016-01-14 17:16:27,843 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:27,842 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:27,843 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791787676_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:27,843 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: F7BD929BE3F047F1A20FCC40B3971F9C : {247A70A8FD924D829389BCDCB113680F={InputName=247A70A8FD924D829389BCDCB113680F}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:27,843 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 247A70A8FD924D829389BCDCB113680F, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:27,843 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791787676_0001_1_00 [F7BD929BE3F047F1A20FCC40B3971F9C]
2016-01-14 17:16:27,843 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791787676_0001_1_00 [F7BD929BE3F047F1A20FCC40B3971F9C]
2016-01-14 17:16:27,843 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791787676_0001_1_00 [F7BD929BE3F047F1A20FCC40B3971F9C]
2016-01-14 17:16:27,843 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791787676_0001_1_00 [F7BD929BE3F047F1A20FCC40B3971F9C]. Starting root input initializers: 1
2016-01-14 17:16:27,844 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791787676_0001_1_00 [F7BD929BE3F047F1A20FCC40B3971F9C] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:27,844 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 247A70A8FD924D829389BCDCB113680F on vertex vertex_1452791787676_0001_1_00 [F7BD929BE3F047F1A20FCC40B3971F9C]
2016-01-14 17:16:27,844 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791787676_0001_1_01 [B2E7CCFF712E46AFB3E8D496A4812B19]
2016-01-14 17:16:27,846 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:27,846 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 247A70A8FD924D829389BCDCB113680F asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:27,846 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:27,847 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:27,847 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/-1_VALUES_RESULTS, applicationId=application_1452791787676_0001, dagName=[739CF7EDADD34C428AED87D51F4EE678/A229578C3EC840FD93BF90F84E0FBCE6] every/(1/1) ...latform/-1_VALUES_RESULTS
2016-01-14 17:16:27,848 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:27,849 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 247A70A8FD924D829389BCDCB113680F on vertex vertex_1452791787676_0001_1_00 [F7BD929BE3F047F1A20FCC40B3971F9C]
2016-01-14 17:16:27,852 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:27,852 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791787676_0001_1_01 [B2E7CCFF712E46AFB3E8D496A4812B19]
2016-01-14 17:16:27,852 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791787676_0001_1_01 [B2E7CCFF712E46AFB3E8D496A4812B19]
2016-01-14 17:16:27,852 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791787676_0001_1_01 [B2E7CCFF712E46AFB3E8D496A4812B19]
2016-01-14 17:16:27,852 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=875781DA3B514A5E8C04B9EE083524E0, vertexId=vertex_1452791787676_0001_1_01 [B2E7CCFF712E46AFB3E8D496A4812B19], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:27,853 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=875781DA3B514A5E8C04B9EE083524E0, vertexId=vertex_1452791787676_0001_1_01 [B2E7CCFF712E46AFB3E8D496A4812B19]
2016-01-14 17:16:27,855 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=875781DA3B514A5E8C04B9EE083524E0, vertexName=B2E7CCFF712E46AFB3E8D496A4812B19, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:27,855 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=875781DA3B514A5E8C04B9EE083524E0, vertexId=vertex_1452791787676_0001_1_01 [B2E7CCFF712E46AFB3E8D496A4812B19]
2016-01-14 17:16:27,857 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791787676_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B2E7CCFF712E46AFB3E8D496A4812B19, vertexId=vertex_1452791787676_0001_1_01, initRequestedTime=1452791787844, initedTime=1452791787857, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:27,857 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791787676_0001_1_01 [B2E7CCFF712E46AFB3E8D496A4812B19] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:27,857 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:27,857 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791787676_0001_1_00 [F7BD929BE3F047F1A20FCC40B3971F9C] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:27,858 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {247A70A8FD924D829389BCDCB113680F=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:27,858 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791787676_0001_1_00 [F7BD929BE3F047F1A20FCC40B3971F9C] parallelism set to 1
2016-01-14 17:16:27,858 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791787676_0001_1_00 [F7BD929BE3F047F1A20FCC40B3971F9C] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:27,859 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791787676_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791787676_0001_1_00, eventCount=1
2016-01-14 17:16:27,859 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:27,859 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791787676_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F7BD929BE3F047F1A20FCC40B3971F9C, vertexId=vertex_1452791787676_0001_1_00, initRequestedTime=1452791787843, initedTime=1452791787859, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:27,860 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791787676_0001_1_00 [F7BD929BE3F047F1A20FCC40B3971F9C] with distanceFromRoot: 0
2016-01-14 17:16:27,860 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791787676_0001_1_00 [F7BD929BE3F047F1A20FCC40B3971F9C] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:27,860 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in F7BD929BE3F047F1A20FCC40B3971F9C
2016-01-14 17:16:27,860 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791787676_0001_1_00 [F7BD929BE3F047F1A20FCC40B3971F9C]
2016-01-14 17:16:27,861 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:27,861 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791787676_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791787676_0001_1_00, startRequestedTime=1452791787860, startedTime=1452791787860
2016-01-14 17:16:27,861 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791787676_0001_1_00 [F7BD929BE3F047F1A20FCC40B3971F9C] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:27,862 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:27,862 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791787676_0001_1][Event:TASK_STARTED]: vertexName=F7BD929BE3F047F1A20FCC40B3971F9C, taskId=task_1452791787676_0001_1_00_000000, scheduledTime=1452791787862, launchTime=1452791787862
2016-01-14 17:16:27,862 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791787676_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:27,863 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791787676_0001_1_00 for vertex: vertex_1452791787676_0001_1_01 [B2E7CCFF712E46AFB3E8D496A4812B19] numStartedSources: 1 numSources: 1
2016-01-14 17:16:27,863 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791787676_0001_1_01 [B2E7CCFF712E46AFB3E8D496A4812B19] with distanceFromRoot: 1
2016-01-14 17:16:27,863 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791787676_0001_1_00 [F7BD929BE3F047F1A20FCC40B3971F9C]
2016-01-14 17:16:27,864 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: F7BD929BE3F047F1A20FCC40B3971F9C in vertex: B2E7CCFF712E46AFB3E8D496A4812B19
2016-01-14 17:16:27,864 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: B2E7CCFF712E46AFB3E8D496A4812B19 with 1 source tasks and 1 pending tasks
2016-01-14 17:16:27,864 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:27,865 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791787676_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791787676_0001_1_01, startRequestedTime=1452791787864, startedTime=1452791787864
2016-01-14 17:16:27,865 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791787676_0001_1_01 [B2E7CCFF712E46AFB3E8D496A4812B19] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:27,865 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [739CF7EDADD34C428AED87D51F4EE678/A229578C3EC840FD93BF90F84E0FBCE6] every/(1/1) ...latform/-1_VALUES_RESULTS, VertexName: F7BD929BE3F047F1A20FCC40B3971F9C, VertexParallelism: 1, TaskAttemptID:attempt_1452791787676_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=247A70A8FD924D829389BCDCB113680F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B2E7CCFF712E46AFB3E8D496A4812B19, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:27,865 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791787676_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:27,866 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:27,866 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791787676_0001, with dag id: dag_1452791787676_0001_1
2016-01-14 17:16:27,868 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:27,869 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:27,869 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:27,869 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791787676_0001_00_000001
2016-01-14 17:16:27,869 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791787676_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:27,871 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791787676_0001_00_000001
2016-01-14 17:16:27,871 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:27,873 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791787676_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:27,873 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:27,874 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791787676_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791787676_0001_00_000001, launchTime=1452791787873
2016-01-14 17:16:27,875 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791787676_0001_00_000001
2016-01-14 17:16:27,876 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791787676_0001_1_00_000000_0] to container: [container_1452791787676_0001_00_000001]
2016-01-14 17:16:27,876 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791787676_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:27,876 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791787676_0001_00_000001 given task: attempt_1452791787676_0001_1_00_000000_0
2016-01-14 17:16:27,877 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791787676_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791787676_0001_1_00_000000_0
2016-01-14 17:16:27,877 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791787676_0001_1_00_000000_0] started. Is using containerId: [container_1452791787676_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:27,877 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791787676_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F7BD929BE3F047F1A20FCC40B3971F9C, taskAttemptId=attempt_1452791787676_0001_1_00_000000_0, startTime=1452791787876, containerId=container_1452791787676_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791787676_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:27,878 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791787676_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:27,878 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:27,878 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791787676_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:27 Starting to run new task attempt: attempt_1452791787676_0001_1_00_000000_0
2016-01-14 17:16:27,878 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791787676_0001_1_00_000000_0
2016-01-14 17:16:27,879 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:27,879 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:27,879 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:27,879 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:27,880 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:27,880 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:27,880 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [739CF7EDADD34C428AED87D51F4EE678/A229578C3EC840FD93BF90F84E0FBCE6] every/(1/1) ...latform/-1_VALUES_RESULTS, VertexName: F7BD929BE3F047F1A20FCC40B3971F9C, VertexParallelism: 1, TaskAttemptID:attempt_1452791787676_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=247A70A8FD924D829389BCDCB113680F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B2E7CCFF712E46AFB3E8D496A4812B19, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:27,880 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:16:27,881 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791787676_0001_1_00_000000_0
2016-01-14 17:16:27,881 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:27,882 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791787676_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:27,882 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:27,882 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=247A70A8FD924D829389BCDCB113680F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:27,883 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=B2E7CCFF712E46AFB3E8D496A4812B19, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:27,882 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:27,883 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:27,883 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:27,884 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: B2E7CCFF712E46AFB3E8D496A4812B19
2016-01-14 17:16:27,883 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 247A70A8FD924D829389BCDCB113680F
2016-01-14 17:16:27,884 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 247A70A8FD924D829389BCDCB113680F
2016-01-14 17:16:27,885 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:27,885 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: B2E7CCFF712E46AFB3E8D496A4812B19
2016-01-14 17:16:27,886 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: B2E7CCFF712E46AFB3E8D496A4812B19
2016-01-14 17:16:27,887 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:27,887 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:27,887 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 247A70A8FD924D829389BCDCB113680F
2016-01-14 17:16:27,888 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 247A70A8FD924D829389BCDCB113680F
2016-01-14 17:16:27,892 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F7BD929BE3F047F1A20FCC40B3971F9C, ordinal: 0
2016-01-14 17:16:27,892 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F7BD929BE3F047F1A20FCC40B3971F9C, mem on start (mb), free: 148, total: 372, max: 672
2016-01-14 17:16:27,893 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:27,893 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:27,893 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:27,893 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:27,893 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:27,893 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:27,893 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:27,893 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:27,894 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:27,894 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 17:16:27,894 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 247A70A8FD924D829389BCDCB113680F, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:27,894 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, B2E7CCFF712E46AFB3E8D496A4812B19, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:27,894 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:27,894 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 247A70A8FD924D829389BCDCB113680F being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:27,895 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:27,895 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:27,895 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 247A70A8FD924D829389BCDCB113680F
2016-01-14 17:16:27,895 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:27,895 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 247A70A8FD924D829389BCDCB113680F
2016-01-14 17:16:27,896 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:27,896 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791787676_0001_1_00_000000_0
2016-01-14 17:16:27,896 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:27,897 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@3c3ad0de
2016-01-14 17:16:27,898 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@3c3ad0de
2016-01-14 17:16:27,898 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:27,898 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:27,899 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:27,899 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 247A70A8FD924D829389BCDCB113680F
2016-01-14 17:16:27,900 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 6D796FBEF59B44048F87C1A1B870FA66
2016-01-14 17:16:27,900 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 6D796FBEF59B44048F87C1A1B870FA66
2016-01-14 17:16:27,900 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@672c5486; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@f615e55; comparator=cascading.tuple.hadoop.util.TupleComparator@41782656; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:27,901 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:27,901 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:27,901 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:28,126 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:28,126 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:28,126 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:28,126 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:28,127 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:28,131 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:28,131 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:28,132 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F7BD929BE3F047F1A20FCC40B3971F9C, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:28,135 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791787676_0001_1_00_000000_0
2016-01-14 17:16:28,135 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F7BD929BE3F047F1A20FCC40B3971F9C, mem on close (mb), free: 358, total: 484, max: 672
2016-01-14 17:16:28,135 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:28,135 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:28,135 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:28,135 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:28,138 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:28 Completed running task attempt: attempt_1452791787676_0001_1_00_000000_0
2016-01-14 17:16:28,144 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791787676_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:28,145 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791787676_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791787676_0001_1_00, eventCount=1
2016-01-14 17:16:28,145 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=211, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=507510784, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791788132, Process_Duration=3, Process_End_Time=1452791788135, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:28,145 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:28,145 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:28,145 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791787676_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F7BD929BE3F047F1A20FCC40B3971F9C, taskAttemptId=attempt_1452791787676_0001_1_00_000000_0, startTime=1452791787876, finishTime=1452791788145, timeTaken=269, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=211, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=507510784, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791788132, Process_Duration=3, Process_End_Time=1452791788135, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:28,145 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791787676_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:28,146 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791787676_0001_1_00_000000_0
2016-01-14 17:16:28,146 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791787676_0001_00_000001
2016-01-14 17:16:28,146 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791787676_0001_1][Event:TASK_FINISHED]: vertexName=F7BD929BE3F047F1A20FCC40B3971F9C, taskId=task_1452791787676_0001_1_00_000000, startTime=1452791787876, finishTime=1452791788146, timeTaken=270, status=SUCCEEDED, successfulAttemptID=attempt_1452791787676_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=211, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=507510784, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791788132, Process_Duration=3, Process_End_Time=1452791788135, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:28,146 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:28,146 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791787676_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:28,146 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:28,147 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791787676_0001_1_00 [F7BD929BE3F047F1A20FCC40B3971F9C] : 1
2016-01-14 17:16:28,147 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791787676_0001_1_00 [F7BD929BE3F047F1A20FCC40B3971F9C]
2016-01-14 17:16:28,147 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:28,148 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791787676_0001_1][Event:VERTEX_FINISHED]: vertexName=F7BD929BE3F047F1A20FCC40B3971F9C, vertexId=vertex_1452791787676_0001_1_00, initRequestedTime=1452791787843, initedTime=1452791787859, startRequestedTime=1452791787860, startedTime=1452791787860, finishTime=1452791788147, timeTaken=287, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=211, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=507510784, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791788132, Process_Duration=3, Process_End_Time=1452791788135, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791787876, firstTasksToStart=[ task_1452791787676_0001_1_00_000000 ], lastTaskFinishTime=1452791788145, lastTasksToFinish=[ task_1452791787676_0001_1_00_000000 ], minTaskDuration=269, maxTaskDuration=269, avgTaskDuration=269.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791787676_0001_1_00_000000 ], longestDurationTasks=[ task_1452791787676_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:28,148 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:28,148 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791787676_0001_1_00 [F7BD929BE3F047F1A20FCC40B3971F9C] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:28,148 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791787676_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:28,149 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791787676_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791787676_0001_00_000001, stoppedTime=1452791788149, exitStatus=0
2016-01-14 17:16:28,149 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791787676_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:28,149 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:28,149 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791787676_0001_1_01 [B2E7CCFF712E46AFB3E8D496A4812B19] attempt: attempt_1452791787676_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:28,150 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: B2E7CCFF712E46AFB3E8D496A4812B19
2016-01-14 17:16:28,150 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791787676_0001_1_01 [B2E7CCFF712E46AFB3E8D496A4812B19]
2016-01-14 17:16:28,150 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791787676_0001_1_00 [F7BD929BE3F047F1A20FCC40B3971F9C] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:28,150 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791787676_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:28,150 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791787676_0001_1][Event:TASK_STARTED]: vertexName=B2E7CCFF712E46AFB3E8D496A4812B19, taskId=task_1452791787676_0001_1_01_000000, scheduledTime=1452791788150, launchTime=1452791788150
2016-01-14 17:16:28,150 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:28,150 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791787676_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:28,152 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791787676_0001_1_01 [B2E7CCFF712E46AFB3E8D496A4812B19]
2016-01-14 17:16:28,152 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791787676_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:28,152 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [739CF7EDADD34C428AED87D51F4EE678/A229578C3EC840FD93BF90F84E0FBCE6] every/(1/1) ...latform/-1_VALUES_RESULTS, VertexName: B2E7CCFF712E46AFB3E8D496A4812B19, VertexParallelism: 1, TaskAttemptID:attempt_1452791787676_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F7BD929BE3F047F1A20FCC40B3971F9C, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=875781DA3B514A5E8C04B9EE083524E0, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:28,152 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791787676_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:28,153 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:28,153 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791787676_0001_00_000002
2016-01-14 17:16:28,153 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791787676_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:28,155 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791787676_0001_00_000002
2016-01-14 17:16:28,155 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:28,156 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:28,156 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791787676_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:28,156 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791787676_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:28,157 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791787676_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:28,156 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791787676_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791787676_0001_00_000002, launchTime=1452791788156
2016-01-14 17:16:28,157 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791787676_0001_00_000001, Exiting
2016-01-14 17:16:28,157 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791787676_0001_00_000001
2016-01-14 17:16:28,157 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791787676_0001_00_000001
2016-01-14 17:16:28,157 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791787676_0001_00_000002
2016-01-14 17:16:28,158 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791787676_0001_1_01_000000_0] to container: [container_1452791787676_0001_00_000002]
2016-01-14 17:16:28,159 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791787676_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:28,159 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791787676_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:28,159 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791787676_0001_00_000001 completed successfully
2016-01-14 17:16:28,159 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791787676_0001_00_000002 given task: attempt_1452791787676_0001_1_01_000000_0
2016-01-14 17:16:28,159 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791787676_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791787676_0001_1_01_000000_0
2016-01-14 17:16:28 Starting to run new task attempt: attempt_1452791787676_0001_1_01_000000_0
2016-01-14 17:16:28,160 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791787676_0001_1_01_000000_0] started. Is using containerId: [container_1452791787676_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:28,160 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791787676_0001_1_01_000000_0
2016-01-14 17:16:28,161 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:28,161 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:28,161 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:28,161 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791787676_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B2E7CCFF712E46AFB3E8D496A4812B19, taskAttemptId=attempt_1452791787676_0001_1_01_000000_0, startTime=1452791788159, containerId=container_1452791787676_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791787676_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:28,161 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:28,161 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:28,162 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:28,162 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:28,162 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [739CF7EDADD34C428AED87D51F4EE678/A229578C3EC840FD93BF90F84E0FBCE6] every/(1/1) ...latform/-1_VALUES_RESULTS, VertexName: B2E7CCFF712E46AFB3E8D496A4812B19, VertexParallelism: 1, TaskAttemptID:attempt_1452791787676_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F7BD929BE3F047F1A20FCC40B3971F9C, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=875781DA3B514A5E8C04B9EE083524E0, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:28,162 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791787676_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:28,162 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:16:28,162 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791787676_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:28,163 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791787676_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:28,163 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791787676_0001_1_01_000000_0
2016-01-14 17:16:28,163 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:28,164 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:28,164 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791787676_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:28,164 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F7BD929BE3F047F1A20FCC40B3971F9C, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:28,165 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=875781DA3B514A5E8C04B9EE083524E0, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:28,165 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:28,165 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:28,165 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F7BD929BE3F047F1A20FCC40B3971F9C
2016-01-14 17:16:28,165 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:28,166 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 875781DA3B514A5E8C04B9EE083524E0
2016-01-14 17:16:28,166 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:28,167 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 875781DA3B514A5E8C04B9EE083524E0
2016-01-14 17:16:28,167 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:28,167 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:28,167 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: F7BD929BE3F047F1A20FCC40B3971F9C
2016-01-14 17:16:28,168 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F7BD929BE3F047F1A20FCC40B3971F9C
2016-01-14 17:16:28,170 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:28,176 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B2E7CCFF712E46AFB3E8D496A4812B19, ordinal: 1
2016-01-14 17:16:28,176 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B2E7CCFF712E46AFB3E8D496A4812B19, mem on start (mb), free: 352, total: 484, max: 672
2016-01-14 17:16:28,176 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:28,176 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:28,177 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:28,182 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:28,182 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 875781DA3B514A5E8C04B9EE083524E0
2016-01-14 17:16:28,183 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:28,183 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:28,183 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:28,183 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:28,183 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:28,183 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:28,184 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:28,184 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 875781DA3B514A5E8C04B9EE083524E0, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:28,184 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F7BD929BE3F047F1A20FCC40B3971F9C, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 17:16:28,184 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:28,184 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F7BD929BE3F047F1A20FCC40B3971F9C being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:28,184 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F7BD929BE3F047F1A20FCC40B3971F9C
2016-01-14 17:16:28,184 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:28,185 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:28,186 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:28,186 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:28,186 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: F7BD929BE3F047F1A20FCC40B3971F9C with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:28,186 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 17:16:28,186 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:28,187 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:28,190 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F7BD929BE3F047F1A20FCC40B3971F9C
2016-01-14 17:16:28,190 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:28,191 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791787676_0001_1_01_000000_0
2016-01-14 17:16:28,191 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791787676_0001_1_00_000000_0_10069, runDuration: 256632, ]
2016-01-14 17:16:28,193 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:28,194 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 6D796FBEF59B44048F87C1A1B870FA66
2016-01-14 17:16:28,194 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_RESULTS"], id: 875781DA3B514A5E8C04B9EE083524E0
2016-01-14 17:16:28,194 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_RESULTS"]
2016-01-14 17:16:28,195 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 6D796FBEF59B44048F87C1A1B870FA66, for 1 inputs
2016-01-14 17:16:28,196 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791787676_0001_1_00_000000_0_10069] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791788196, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 17:16:28,196 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : F7BD929BE3F047F1A20FCC40B3971F9C
2016-01-14 17:16:28,196 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:28,196 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [F7BD929BE3F047F1A20FCC40B3971F9C] #12 in 2ms
2016-01-14 17:16:28,197 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [F7BD929BE3F047F1A20FCC40B3971F9C] #12, status:false, isInterrupted:false
2016-01-14 17:16:28,197 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:28,198 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:28,198 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:28,198 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:28,198 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:28,198 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : F7BD929BE3F047F1A20FCC40B3971F9C
2016-01-14 17:16:28,198 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:28,199 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B2E7CCFF712E46AFB3E8D496A4812B19, all 1 inputs ready in: 00:00:00.004
2016-01-14 17:16:28,199 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@412bb519; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@1c7e9dbc
2016-01-14 17:16:28,200 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:28,203 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(92)) - caught throwable
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:28,205 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:28,205 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:28,206 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791787676_0001_1_01_000000_0
2016-01-14 17:16:28,206 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791787676_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:28,206 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 17:16:28,207 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 17:16:28,208 INFO  task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1452791787676_0001_1_01_000000_0
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:28,213 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=528, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=507510784, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791788199, Process_Duration=9, Process_End_Time=1452791788208, Read_Duration=0, Tuples_Read=1]]
2016-01-14 17:16:28,213 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791787676_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B2E7CCFF712E46AFB3E8D496A4812B19, taskAttemptId=attempt_1452791787676_0001_1_01_000000_0, startTime=1452791788159, finishTime=1452791788213, timeTaken=54, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=528, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=507510784, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791788199, Process_Duration=9, Process_End_Time=1452791788208, Read_Duration=0, Tuples_Read=1
2016-01-14 17:16:28,213 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791787676_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-01-14 17:16:28,214 INFO  impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1452791787676_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-01-14 17:16:28,214 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791787676_0001_1][Event:TASK_FINISHED]: vertexName=B2E7CCFF712E46AFB3E8D496A4812B19, taskId=task_1452791787676_0001_1_01_000000, startTime=1452791788159, finishTime=1452791788214, timeTaken=55, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
], counters=Counters: 0
2016-01-14 17:16:28,214 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791787676_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-01-14 17:16:28,214 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:28,213 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B2E7CCFF712E46AFB3E8D496A4812B19, mem on close (mb), free: 349, total: 484, max: 672
2016-01-14 17:16:28,214 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791787676_0001_1_01 [B2E7CCFF712E46AFB3E8D496A4812B19] : 1
2016-01-14 17:16:28,213 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:28,218 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=B2E7CCFF712E46AFB3E8D496A4812B19, index=1
2016-01-14 17:16:28,219 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:28,218 INFO  impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1452791787676_0001_1_01 [B2E7CCFF712E46AFB3E8D496A4812B19] because task failed: task_1452791787676_0001_1_01_000000
2016-01-14 17:16:28,219 INFO  impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1452791787676_0001_1_01 [B2E7CCFF712E46AFB3E8D496A4812B19] due to trigger: OWN_TASK_FAILURE
2016-01-14 17:16:28,219 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-01-14 17:16:28,219 INFO  impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1452791787676_0001_1_01 [B2E7CCFF712E46AFB3E8D496A4812B19]
2016-01-14 17:16:28,220 INFO  impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=875781DA3B514A5E8C04B9EE083524E0, vertexId=vertex_1452791787676_0001_1_01 [B2E7CCFF712E46AFB3E8D496A4812B19]
2016-01-14 17:16:28,221 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:28,221 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791787676_0001_1][Event:VERTEX_FINISHED]: vertexName=B2E7CCFF712E46AFB3E8D496A4812B19, vertexId=vertex_1452791787676_0001_1_01, initRequestedTime=1452791787844, initedTime=1452791787857, startRequestedTime=1452791787864, startedTime=1452791787864, finishTime=1452791788221, timeTaken=357, status=FAILED, diagnostics=Task failed, taskId=task_1452791787676_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1452791787676_0001_1_01 [B2E7CCFF712E46AFB3E8D496A4812B19] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1452791788159, firstTasksToStart=[ task_1452791787676_0001_1_01_000000 ], lastTaskFinishTime=1452791788213, lastTasksToFinish=[ task_1452791787676_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[  ], longestDurationTasks=[  ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-01-14 17:16:28,221 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791787676_0001_1_01 [B2E7CCFF712E46AFB3E8D496A4812B19] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-01-14 17:16:28,222 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791787676_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791787676_0001_00_000002, stoppedTime=1452791788222, exitStatus=0
2016-01-14 17:16:28,222 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:28,222 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791787676_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:28,222 INFO  node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1452791787676_0001_1_01_000000_0 failed: true container: container_1452791787676_0001_00_000002 numFailedTAs: 0
2016-01-14 17:16:28,222 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791787676_0001_1_01 [B2E7CCFF712E46AFB3E8D496A4812B19] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-01-14 17:16:28,222 INFO  impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-01-14 17:16:28,222 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791787676_0001_1
2016-01-14 17:16:28,223 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1452791787676_0001_1_00 [F7BD929BE3F047F1A20FCC40B3971F9C]
2016-01-14 17:16:28,223 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 875781DA3B514A5E8C04B9EE083524E0 for vertex: vertex_1452791787676_0001_1_01 [B2E7CCFF712E46AFB3E8D496A4812B19]
2016-01-14 17:16:28,226 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791787676_0001_1, queueSize=0
2016-01-14 17:16:28,226 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:28,226 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791787676_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791787676_0001_1, startTime=1452791787842, finishTime=1452791788223, timeTaken=381, status=FAILED, diagnostics=Vertex failed, vertexName=B2E7CCFF712E46AFB3E8D496A4812B19, vertexId=vertex_1452791787676_0001_1_01, diagnostics=[Task failed, taskId=task_1452791787676_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1452791787676_0001_1_01 [B2E7CCFF712E46AFB3E8D496A4812B19] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=211, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=507510784, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791788132, Process_Duration=3, Process_End_Time=1452791788135, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:28,226 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791787676_0001_1 finished with state: FAILED
2016-01-14 17:16:28,226 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791787676_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:28,227 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791787676_0001_00_000002. Relying on regular task shutdown for it to end
2016-01-14 17:16:28 Completed Dag: dag_1452791787676_0001_1
2016-01-14 17:16:28,227 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791787676_0001_1, dagState=FAILED
2016-01-14 17:16:28,227 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791787676_0001_1_post
2016-01-14 17:16:28,227 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:28,227 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:28,227 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791787676_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:28,229 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: F7BD929BE3F047F1A20FCC40B3971F9C
2016-01-14 17:16:28,229 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:28,230 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:28,230 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:28,230 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:28,230 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:28,230 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=B2E7CCFF712E46AFB3E8D496A4812B19, sourceVertex=F7BD929BE3F047F1A20FCC40B3971F9C
2016-01-14 17:16:28,230 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=B2E7CCFF712E46AFB3E8D496A4812B19, sourceVertex=875781DA3B514A5E8C04B9EE083524E0
2016-01-14 17:16:28,230 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791787676_0001_00_000002
2016-01-14 17:16:28,230 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791787676_0001_00_000002
2016-01-14 17:16:28,230 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791787676_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-01-14 17:16:28,231 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1452791787676_0001_00_000002 completed but with errors
2016-01-14 17:16:28,231 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1452791787676_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-01-14 17:16:28,231 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791787676_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:28,231 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791787676_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-01-14 17:16:28,234 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/-1_VALUES_RESULTS, applicationId=application_1452791787676_0001
2016-01-14 17:16:28,236 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:28,236 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:28,236 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:28,236 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:28,236 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:28,237 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:28,237 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:28,238 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:28,238 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:28,238 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:28,239 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:28,239 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:28,239 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:28,245 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/739CF7EDADD34C428AED87D51F4EE678/A229578C3EC840FD93BF90F84E0FBCE6/.tez/application_1452791787676_0001
2016-01-14 17:16:28,245 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping all jobs
2016-01-14 17:16:28,246 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping: (1/1) ...latform/-1_VALUES_RESULTS
2016-01-14 17:16:28,246 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopped all jobs
2016-01-14 17:16:28,247 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.590
2016-01-14 17:16:28,247 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_VALUES_RESULTS
2016-01-14 17:16:28,282 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:28,283 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:28,327 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.019
2016-01-14 17:16:28,330 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.022
2016-01-14 17:16:28,330 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:28,330 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:28,330 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:28,345 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:28,351 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_ALL
2016-01-14 17:16:28,396 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:28,396 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:28,397 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_ALL"]
2016-01-14 17:16:28,397 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:28,397 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:28,398 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:28,399 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...blyplatform/-1_VALUES_ALL
2016-01-14 17:16:28,399 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:28,413 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:28,417 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:28,418 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:28,418 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:28,427 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/031BE6876EC1423BA4E94963FC78E5FC/AB7FE986F66C4C22BDE9616793FD5944/.tez/application_1452791788418_0001 doesn't exist and is created
2016-01-14 17:16:28,443 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:28,444 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:28,445 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/031BE6876EC1423BA4E94963FC78E5FC/AB7FE986F66C4C22BDE9616793FD5944/.tez/application_1452791788418_0001
2016-01-14 17:16:28,448 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791788418_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:28,458 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:28,459 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:28,461 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:28,461 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:28,462 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/031BE6876EC1423BA4E94963FC78E5FC/AB7FE986F66C4C22BDE9616793FD5944/.tez/application_1452791788418_0001/localmode-log-dir
2016-01-14 17:16:28,462 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/031BE6876EC1423BA4E94963FC78E5FC/AB7FE986F66C4C22BDE9616793FD5944/.tez/application_1452791788418_0001/localmode-log-dir/history.txt.appattempt_1452791788418_0001_000000, maxErrors=10
2016-01-14 17:16:28,462 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:28,462 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791788418_0001_000000, appSubmitTime=1452791788447, launchTime=1452791788447
2016-01-14 17:16:28,468 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:28,469 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:59140
2016-01-14 17:16:28,472 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:28,472 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:28,473 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:28,474 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:28,480 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791788418_0001_000000, startTime=1452791788480
2016-01-14 17:16:28,480 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:28,480 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:28,544 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:28,544 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:28,544 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...blyplatform/-1_VALUES_ALL, applicationId=application_1452791788418_0001, dagName=[031BE6876EC1423BA4E94963FC78E5FC/AB7FE986F66C4C22BDE9616793FD5944] every/(1/1) ...blyplatform/-1_VALUES_ALL
2016-01-14 17:16:28,563 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [031BE6876EC1423BA4E94963FC78E5FC/AB7FE986F66C4C22BDE9616793FD5944] every/(1/1) ...blyplatform/-1_VALUES_ALL
2016-01-14 17:16:28,564 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791788418_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/031BE6876EC1423BA4E94963FC78E5FC/AB7FE986F66C4C22BDE9616793FD5944/.tez/application_1452791788418_0001/localmode-log-dir/dag_1452791788418_0001_1.dot
2016-01-14 17:16:28,564 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/031BE6876EC1423BA4E94963FC78E5FC/AB7FE986F66C4C22BDE9616793FD5944/.tez/application_1452791788418_0001/localmode-log-dir/dag_1452791788418_0001_1-tez-dag.pb.txt
2016-01-14 17:16:28,571 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791788418_0001_1
2016-01-14 17:16:28,571 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:28,571 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [031BE6876EC1423BA4E94963FC78E5FC/AB7FE986F66C4C22BDE9616793FD5944] every/(1/1) ...blyplatform/-1_VALUES_ALL
2016-01-14 17:16:28 Running Dag: dag_1452791788418_0001_1
2016-01-14 17:16:28,584 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:28,584 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791788418_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791788418_0001_1, submitTime=1452791788563
2016-01-14 17:16:28,585 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex E07E40F0BB0E4EC3B29CFAF1D826791A
2016-01-14 17:16:28,589 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:28,590 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:28,590 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791788418_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791788418_0001_1, initTime=1452791788584
2016-01-14 17:16:28,590 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791788418_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:28,591 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791788418_0001_1][Event:DAG_STARTED]: dagID=dag_1452791788418_0001_1, startTime=1452791788591
2016-01-14 17:16:28,591 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:28,591 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:28,591 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791788418_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:28,592 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C8192DFE78E04E2F93D37A91436CEB63 : {A8FFB0B5627D44A6A2582D13147CC796={InputName=A8FFB0B5627D44A6A2582D13147CC796}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:28,592 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: A8FFB0B5627D44A6A2582D13147CC796, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:28,592 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791788418_0001_1_00 [C8192DFE78E04E2F93D37A91436CEB63]
2016-01-14 17:16:28,592 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791788418_0001_1_00 [C8192DFE78E04E2F93D37A91436CEB63]
2016-01-14 17:16:28,592 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791788418_0001_1_00 [C8192DFE78E04E2F93D37A91436CEB63]
2016-01-14 17:16:28,592 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791788418_0001_1_00 [C8192DFE78E04E2F93D37A91436CEB63]. Starting root input initializers: 1
2016-01-14 17:16:28,595 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791788418_0001_1_00 [C8192DFE78E04E2F93D37A91436CEB63] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:28,595 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791788418_0001_1_01 [E07E40F0BB0E4EC3B29CFAF1D826791A]
2016-01-14 17:16:28,596 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...blyplatform/-1_VALUES_ALL, applicationId=application_1452791788418_0001, dagName=[031BE6876EC1423BA4E94963FC78E5FC/AB7FE986F66C4C22BDE9616793FD5944] every/(1/1) ...blyplatform/-1_VALUES_ALL
2016-01-14 17:16:28,598 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:28,598 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791788418_0001_1_01 [E07E40F0BB0E4EC3B29CFAF1D826791A]
2016-01-14 17:16:28,599 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791788418_0001_1_01 [E07E40F0BB0E4EC3B29CFAF1D826791A]
2016-01-14 17:16:28,599 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791788418_0001_1_01 [E07E40F0BB0E4EC3B29CFAF1D826791A]
2016-01-14 17:16:28,599 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=628E81455DFF41D98643A3D818C2057F, vertexId=vertex_1452791788418_0001_1_01 [E07E40F0BB0E4EC3B29CFAF1D826791A], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:28,599 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=628E81455DFF41D98643A3D818C2057F, vertexId=vertex_1452791788418_0001_1_01 [E07E40F0BB0E4EC3B29CFAF1D826791A]
2016-01-14 17:16:28,598 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: A8FFB0B5627D44A6A2582D13147CC796 on vertex vertex_1452791788418_0001_1_00 [C8192DFE78E04E2F93D37A91436CEB63]
2016-01-14 17:16:28,601 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=628E81455DFF41D98643A3D818C2057F, vertexName=E07E40F0BB0E4EC3B29CFAF1D826791A, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:28,601 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=628E81455DFF41D98643A3D818C2057F, vertexId=vertex_1452791788418_0001_1_01 [E07E40F0BB0E4EC3B29CFAF1D826791A]
2016-01-14 17:16:28,603 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:28,603 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791788418_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E07E40F0BB0E4EC3B29CFAF1D826791A, vertexId=vertex_1452791788418_0001_1_01, initRequestedTime=1452791788595, initedTime=1452791788603, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:28,603 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791788418_0001_1_01 [E07E40F0BB0E4EC3B29CFAF1D826791A] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:28,608 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:28,608 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input A8FFB0B5627D44A6A2582D13147CC796 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:28,610 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:28,610 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:28,613 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:28,613 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: A8FFB0B5627D44A6A2582D13147CC796 on vertex vertex_1452791788418_0001_1_00 [C8192DFE78E04E2F93D37A91436CEB63]
2016-01-14 17:16:28,614 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791788418_0001, with dag id: dag_1452791788418_0001_1
2016-01-14 17:16:28,614 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791788418_0001_1_00 [C8192DFE78E04E2F93D37A91436CEB63] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:28,614 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {A8FFB0B5627D44A6A2582D13147CC796=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:28,615 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791788418_0001_1_00 [C8192DFE78E04E2F93D37A91436CEB63] parallelism set to 1
2016-01-14 17:16:28,615 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791788418_0001_1_00 [C8192DFE78E04E2F93D37A91436CEB63] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:28,615 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791788418_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791788418_0001_1_00, eventCount=1
2016-01-14 17:16:28,615 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791788418_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C8192DFE78E04E2F93D37A91436CEB63, vertexId=vertex_1452791788418_0001_1_00, initRequestedTime=1452791788591, initedTime=1452791788615, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:28,615 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791788418_0001_1_00 [C8192DFE78E04E2F93D37A91436CEB63] with distanceFromRoot: 0
2016-01-14 17:16:28,615 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791788418_0001_1_00 [C8192DFE78E04E2F93D37A91436CEB63] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:28,615 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:28,615 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C8192DFE78E04E2F93D37A91436CEB63
2016-01-14 17:16:28,616 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791788418_0001_1_00 [C8192DFE78E04E2F93D37A91436CEB63]
2016-01-14 17:16:28,616 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791788418_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791788418_0001_1_00, startRequestedTime=1452791788615, startedTime=1452791788615
2016-01-14 17:16:28,616 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:28,616 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791788418_0001_1_00 [C8192DFE78E04E2F93D37A91436CEB63] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:28,617 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791788418_0001_1][Event:TASK_STARTED]: vertexName=C8192DFE78E04E2F93D37A91436CEB63, taskId=task_1452791788418_0001_1_00_000000, scheduledTime=1452791788617, launchTime=1452791788617
2016-01-14 17:16:28,617 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791788418_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:28,617 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791788418_0001_1_00 for vertex: vertex_1452791788418_0001_1_01 [E07E40F0BB0E4EC3B29CFAF1D826791A] numStartedSources: 1 numSources: 1
2016-01-14 17:16:28,617 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791788418_0001_1_01 [E07E40F0BB0E4EC3B29CFAF1D826791A] with distanceFromRoot: 1
2016-01-14 17:16:28,617 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791788418_0001_1_00 [C8192DFE78E04E2F93D37A91436CEB63]
2016-01-14 17:16:28,618 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: C8192DFE78E04E2F93D37A91436CEB63 in vertex: E07E40F0BB0E4EC3B29CFAF1D826791A
2016-01-14 17:16:28,618 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: E07E40F0BB0E4EC3B29CFAF1D826791A with 1 source tasks and 1 pending tasks
2016-01-14 17:16:28,618 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:28,619 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:28,620 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791788418_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791788418_0001_1_01, startRequestedTime=1452791788618, startedTime=1452791788618
2016-01-14 17:16:28,620 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791788418_0001_1_01 [E07E40F0BB0E4EC3B29CFAF1D826791A] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:28,620 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [031BE6876EC1423BA4E94963FC78E5FC/AB7FE986F66C4C22BDE9616793FD5944] every/(1/1) ...blyplatform/-1_VALUES_ALL, VertexName: C8192DFE78E04E2F93D37A91436CEB63, VertexParallelism: 1, TaskAttemptID:attempt_1452791788418_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A8FFB0B5627D44A6A2582D13147CC796, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E07E40F0BB0E4EC3B29CFAF1D826791A, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:28,621 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791788418_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:28,621 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:28,630 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:28,631 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:28,631 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:28,631 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791788418_0001_00_000001
2016-01-14 17:16:28,631 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791788418_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:28,633 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791788418_0001_00_000001
2016-01-14 17:16:28,633 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:28,635 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791788418_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:28,635 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:28,636 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791788418_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791788418_0001_00_000001, launchTime=1452791788635
2016-01-14 17:16:28,637 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791788418_0001_00_000001
2016-01-14 17:16:28,637 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791788418_0001_1_00_000000_0] to container: [container_1452791788418_0001_00_000001]
2016-01-14 17:16:28,637 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791788418_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:28,638 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791788418_0001_1_00_000000_0] started. Is using containerId: [container_1452791788418_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:28,638 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791788418_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C8192DFE78E04E2F93D37A91436CEB63, taskAttemptId=attempt_1452791788418_0001_1_00_000000_0, startTime=1452791788638, containerId=container_1452791788418_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791788418_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:28,638 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791788418_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:28,638 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791788418_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:28,639 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:28,639 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791788418_0001_00_000001 given task: attempt_1452791788418_0001_1_00_000000_0
2016-01-14 17:16:28,639 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791788418_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791788418_0001_1_00_000000_0
2016-01-14 17:16:28 Starting to run new task attempt: attempt_1452791788418_0001_1_00_000000_0
2016-01-14 17:16:28,639 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791788418_0001_1_00_000000_0
2016-01-14 17:16:28,640 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:28,640 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:28,640 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:28,640 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:28,640 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:28,640 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:28,640 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [031BE6876EC1423BA4E94963FC78E5FC/AB7FE986F66C4C22BDE9616793FD5944] every/(1/1) ...blyplatform/-1_VALUES_ALL, VertexName: C8192DFE78E04E2F93D37A91436CEB63, VertexParallelism: 1, TaskAttemptID:attempt_1452791788418_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A8FFB0B5627D44A6A2582D13147CC796, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E07E40F0BB0E4EC3B29CFAF1D826791A, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:28,640 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:16:28,642 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791788418_0001_1_00_000000_0
2016-01-14 17:16:28,642 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:28,642 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:28,643 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791788418_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:28,643 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=A8FFB0B5627D44A6A2582D13147CC796, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:28,643 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:28,644 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: A8FFB0B5627D44A6A2582D13147CC796
2016-01-14 17:16:28,644 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: A8FFB0B5627D44A6A2582D13147CC796
2016-01-14 17:16:28,647 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:28,650 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:28,650 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: A8FFB0B5627D44A6A2582D13147CC796
2016-01-14 17:16:28,650 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: A8FFB0B5627D44A6A2582D13147CC796
2016-01-14 17:16:28,651 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:28,652 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=E07E40F0BB0E4EC3B29CFAF1D826791A, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:28,653 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:28,653 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: E07E40F0BB0E4EC3B29CFAF1D826791A
2016-01-14 17:16:28,655 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:28,656 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: E07E40F0BB0E4EC3B29CFAF1D826791A
2016-01-14 17:16:28,656 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: E07E40F0BB0E4EC3B29CFAF1D826791A
2016-01-14 17:16:28,656 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C8192DFE78E04E2F93D37A91436CEB63, ordinal: 0
2016-01-14 17:16:28,657 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C8192DFE78E04E2F93D37A91436CEB63, mem on start (mb), free: 273, total: 484, max: 672
2016-01-14 17:16:28,657 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:28,657 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:28,657 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:28,657 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:28,657 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:28,657 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:28,657 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:28,657 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:28,658 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:28,658 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 17:16:28,658 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, A8FFB0B5627D44A6A2582D13147CC796, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:28,658 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, E07E40F0BB0E4EC3B29CFAF1D826791A, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:28,658 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:28,658 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: A8FFB0B5627D44A6A2582D13147CC796
2016-01-14 17:16:28,658 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:28,658 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: A8FFB0B5627D44A6A2582D13147CC796
2016-01-14 17:16:28,659 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: A8FFB0B5627D44A6A2582D13147CC796 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:28,659 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:28,659 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:28,659 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:28,660 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791788418_0001_1_00_000000_0
2016-01-14 17:16:28,663 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:28,663 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: A8FFB0B5627D44A6A2582D13147CC796
2016-01-14 17:16:28,663 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: ED8AE7EFE7274884B88AACF438D8A195
2016-01-14 17:16:28,664 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] ED8AE7EFE7274884B88AACF438D8A195
2016-01-14 17:16:28,664 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@374ddafe; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@5864760f; comparator=cascading.tuple.hadoop.util.TupleComparator@2b83001f; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:28,665 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:28,665 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:28,665 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:28,666 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:28,847 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@7ab37ac1
2016-01-14 17:16:28,848 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@7ab37ac1
2016-01-14 17:16:28,848 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:28,848 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:28,859 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:28,859 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:28,859 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:28,859 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:28,859 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:28,860 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:28,860 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:28,860 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C8192DFE78E04E2F93D37A91436CEB63, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:28,864 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791788418_0001_1_00_000000_0
2016-01-14 17:16:28,864 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C8192DFE78E04E2F93D37A91436CEB63, mem on close (mb), free: 480, total: 603, max: 672
2016-01-14 17:16:28,864 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:28,864 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:28,864 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:28,864 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:28,867 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:28 Completed running task attempt: attempt_1452791788418_0001_1_00_000000_0
2016-01-14 17:16:28,870 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791788418_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:28,871 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791788418_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791788418_0001_1_00, eventCount=1
2016-01-14 17:16:28,871 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=179, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=632815616, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791788860, Process_Duration=4, Process_End_Time=1452791788864, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:28,872 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791788418_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C8192DFE78E04E2F93D37A91436CEB63, taskAttemptId=attempt_1452791788418_0001_1_00_000000_0, startTime=1452791788638, finishTime=1452791788871, timeTaken=233, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=179, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=632815616, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791788860, Process_Duration=4, Process_End_Time=1452791788864, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:28,872 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791788418_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:28,872 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:28,872 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791788418_0001_1_00_000000_0
2016-01-14 17:16:28,872 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:28,872 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:28,872 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:28,873 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:28,873 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791788418_0001_1][Event:TASK_FINISHED]: vertexName=C8192DFE78E04E2F93D37A91436CEB63, taskId=task_1452791788418_0001_1_00_000000, startTime=1452791788638, finishTime=1452791788872, timeTaken=234, status=SUCCEEDED, successfulAttemptID=attempt_1452791788418_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=179, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=632815616, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791788860, Process_Duration=4, Process_End_Time=1452791788864, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:28,874 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791788418_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:28,874 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791788418_0001_1_00 [C8192DFE78E04E2F93D37A91436CEB63] : 1
2016-01-14 17:16:28,874 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791788418_0001_1_00 [C8192DFE78E04E2F93D37A91436CEB63]
2016-01-14 17:16:28,874 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791788418_0001_00_000001
2016-01-14 17:16:28,875 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:28,875 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791788418_0001_1][Event:VERTEX_FINISHED]: vertexName=C8192DFE78E04E2F93D37A91436CEB63, vertexId=vertex_1452791788418_0001_1_00, initRequestedTime=1452791788591, initedTime=1452791788615, startRequestedTime=1452791788615, startedTime=1452791788615, finishTime=1452791788874, timeTaken=259, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=179, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=632815616, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791788860, Process_Duration=4, Process_End_Time=1452791788864, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791788638, firstTasksToStart=[ task_1452791788418_0001_1_00_000000 ], lastTaskFinishTime=1452791788871, lastTasksToFinish=[ task_1452791788418_0001_1_00_000000 ], minTaskDuration=233, maxTaskDuration=233, avgTaskDuration=233.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791788418_0001_1_00_000000 ], longestDurationTasks=[ task_1452791788418_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:28,875 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791788418_0001_1_00 [C8192DFE78E04E2F93D37A91436CEB63] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:28,876 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791788418_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:28,876 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791788418_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791788418_0001_00_000001, stoppedTime=1452791788876, exitStatus=0
2016-01-14 17:16:28,876 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791788418_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:28,876 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:28,876 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791788418_0001_1_01 [E07E40F0BB0E4EC3B29CFAF1D826791A] attempt: attempt_1452791788418_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:28,876 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: E07E40F0BB0E4EC3B29CFAF1D826791A
2016-01-14 17:16:28,876 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791788418_0001_1_01 [E07E40F0BB0E4EC3B29CFAF1D826791A]
2016-01-14 17:16:28,877 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791788418_0001_1_00 [C8192DFE78E04E2F93D37A91436CEB63] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:28,877 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791788418_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:28,877 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791788418_0001_1][Event:TASK_STARTED]: vertexName=E07E40F0BB0E4EC3B29CFAF1D826791A, taskId=task_1452791788418_0001_1_01_000000, scheduledTime=1452791788877, launchTime=1452791788877
2016-01-14 17:16:28,878 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791788418_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:28,878 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791788418_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:28,878 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791788418_0001_1_01 [E07E40F0BB0E4EC3B29CFAF1D826791A]
2016-01-14 17:16:28,878 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [031BE6876EC1423BA4E94963FC78E5FC/AB7FE986F66C4C22BDE9616793FD5944] every/(1/1) ...blyplatform/-1_VALUES_ALL, VertexName: E07E40F0BB0E4EC3B29CFAF1D826791A, VertexParallelism: 1, TaskAttemptID:attempt_1452791788418_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C8192DFE78E04E2F93D37A91436CEB63, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=628E81455DFF41D98643A3D818C2057F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:28,878 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:28,879 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791788418_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:28,879 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:28,880 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791788418_0001_00_000002
2016-01-14 17:16:28,880 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791788418_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:28,881 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791788418_0001_00_000002
2016-01-14 17:16:28,881 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:28,882 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791788418_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:28,882 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:28,882 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791788418_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791788418_0001_00_000002, launchTime=1452791788882
2016-01-14 17:16:28,883 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791788418_0001_00_000002
2016-01-14 17:16:28,883 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791788418_0001_1_01_000000_0] to container: [container_1452791788418_0001_00_000002]
2016-01-14 17:16:28,883 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791788418_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:28,884 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791788418_0001_1_01_000000_0] started. Is using containerId: [container_1452791788418_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:28,884 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791788418_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E07E40F0BB0E4EC3B29CFAF1D826791A, taskAttemptId=attempt_1452791788418_0001_1_01_000000_0, startTime=1452791788884, containerId=container_1452791788418_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791788418_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:28,884 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791788418_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:28,884 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791788418_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:28,884 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791788418_0001_00_000002 given task: attempt_1452791788418_0001_1_01_000000_0
2016-01-14 17:16:28,885 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791788418_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791788418_0001_1_01_000000_0
2016-01-14 17:16:28,884 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:28,885 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791788418_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:28,885 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791788418_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:28,885 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791788418_0001_00_000001, Exiting
2016-01-14 17:16:28,885 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791788418_0001_00_000001
2016-01-14 17:16:28,885 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791788418_0001_00_000001
2016-01-14 17:16:28 Starting to run new task attempt: attempt_1452791788418_0001_1_01_000000_0
2016-01-14 17:16:28,886 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791788418_0001_1_01_000000_0
2016-01-14 17:16:28,886 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:28,886 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:28,887 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:28,886 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791788418_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:28,887 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:28,887 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791788418_0001_00_000001 completed successfully
2016-01-14 17:16:28,888 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:28,888 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:28,888 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [031BE6876EC1423BA4E94963FC78E5FC/AB7FE986F66C4C22BDE9616793FD5944] every/(1/1) ...blyplatform/-1_VALUES_ALL, VertexName: E07E40F0BB0E4EC3B29CFAF1D826791A, VertexParallelism: 1, TaskAttemptID:attempt_1452791788418_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C8192DFE78E04E2F93D37A91436CEB63, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=628E81455DFF41D98643A3D818C2057F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:28,888 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791788418_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:28,888 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:16:28,890 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791788418_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:28,890 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791788418_0001_1_01_000000_0
2016-01-14 17:16:28,890 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:28,890 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:28,891 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C8192DFE78E04E2F93D37A91436CEB63, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:28,891 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=628E81455DFF41D98643A3D818C2057F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:28,892 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:28,891 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:28,892 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C8192DFE78E04E2F93D37A91436CEB63
2016-01-14 17:16:28,893 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:28,893 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 628E81455DFF41D98643A3D818C2057F
2016-01-14 17:16:28,893 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:28,893 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 628E81455DFF41D98643A3D818C2057F
2016-01-14 17:16:28,894 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:28,895 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:28,895 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: C8192DFE78E04E2F93D37A91436CEB63
2016-01-14 17:16:28,895 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C8192DFE78E04E2F93D37A91436CEB63
2016-01-14 17:16:28,895 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:28,903 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:28,903 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 628E81455DFF41D98643A3D818C2057F
2016-01-14 17:16:28,904 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E07E40F0BB0E4EC3B29CFAF1D826791A, ordinal: 1
2016-01-14 17:16:28,904 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E07E40F0BB0E4EC3B29CFAF1D826791A, mem on start (mb), free: 474, total: 603, max: 672
2016-01-14 17:16:28,905 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:28,905 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:28,905 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:28,905 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:28,905 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:28,905 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:28,905 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:28,905 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:28,905 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:28,905 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:28,906 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 628E81455DFF41D98643A3D818C2057F, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:28,906 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C8192DFE78E04E2F93D37A91436CEB63, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 17:16:28,906 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:28,906 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C8192DFE78E04E2F93D37A91436CEB63
2016-01-14 17:16:28,906 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C8192DFE78E04E2F93D37A91436CEB63 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:28,907 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:28,907 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:28,908 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: C8192DFE78E04E2F93D37A91436CEB63 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:28,908 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 17:16:28,908 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:28,909 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:28,910 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C8192DFE78E04E2F93D37A91436CEB63
2016-01-14 17:16:28,910 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:28,910 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:28,911 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:28,911 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791788418_0001_1_01_000000_0
2016-01-14 17:16:28,911 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791788418_0001_1_00_000000_0_10075, runDuration: 215540, ]
2016-01-14 17:16:28,912 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791788418_0001_1_00_000000_0_10075] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791788912, TimeTaken=0, Rate=0.00 MB/s
2016-01-14 17:16:28,912 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : C8192DFE78E04E2F93D37A91436CEB63
2016-01-14 17:16:28,913 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:28,913 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [C8192DFE78E04E2F93D37A91436CEB63] #13 in 1ms
2016-01-14 17:16:28,913 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [C8192DFE78E04E2F93D37A91436CEB63] #13, status:false, isInterrupted:false
2016-01-14 17:16:28,914 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:28,915 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:28,915 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:28,915 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:28,915 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:28,915 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : C8192DFE78E04E2F93D37A91436CEB63
2016-01-14 17:16:28,915 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:28,918 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:28,918 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: ED8AE7EFE7274884B88AACF438D8A195
2016-01-14 17:16:28,918 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_ALL"], id: 628E81455DFF41D98643A3D818C2057F
2016-01-14 17:16:28,919 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_ALL"]
2016-01-14 17:16:28,919 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] ED8AE7EFE7274884B88AACF438D8A195, for 1 inputs
2016-01-14 17:16:28,919 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E07E40F0BB0E4EC3B29CFAF1D826791A, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:28,920 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@13c57e38; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@ae198ac
2016-01-14 17:16:28,920 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:28,921 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(92)) - caught throwable
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:28,922 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:28,922 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:28,922 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791788418_0001_1_01_000000_0
2016-01-14 17:16:28,922 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791788418_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:28,923 INFO  task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1452791788418_0001_1_01_000000_0
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:28,927 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791788418_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E07E40F0BB0E4EC3B29CFAF1D826791A, taskAttemptId=attempt_1452791788418_0001_1_01_000000_0, startTime=1452791788884, finishTime=1452791788927, timeTaken=43, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=632815616, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791788919, Process_Duration=4, Process_End_Time=1452791788923, Read_Duration=0, Tuples_Read=1
2016-01-14 17:16:28,928 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791788418_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-01-14 17:16:28,928 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:28,928 INFO  impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1452791788418_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-01-14 17:16:28,928 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791788418_0001_1][Event:TASK_FINISHED]: vertexName=E07E40F0BB0E4EC3B29CFAF1D826791A, taskId=task_1452791788418_0001_1_01_000000, startTime=1452791788884, finishTime=1452791788928, timeTaken=44, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
], counters=Counters: 0
2016-01-14 17:16:28,928 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791788418_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-01-14 17:16:28,929 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:28,929 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791788418_0001_1_01 [E07E40F0BB0E4EC3B29CFAF1D826791A] : 1
2016-01-14 17:16:28,929 INFO  impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1452791788418_0001_1_01 [E07E40F0BB0E4EC3B29CFAF1D826791A] because task failed: task_1452791788418_0001_1_01_000000
2016-01-14 17:16:28,929 INFO  impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1452791788418_0001_1_01 [E07E40F0BB0E4EC3B29CFAF1D826791A] due to trigger: OWN_TASK_FAILURE
2016-01-14 17:16:28,929 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-01-14 17:16:28,929 INFO  impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1452791788418_0001_1_01 [E07E40F0BB0E4EC3B29CFAF1D826791A]
2016-01-14 17:16:28,929 INFO  impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=628E81455DFF41D98643A3D818C2057F, vertexId=vertex_1452791788418_0001_1_01 [E07E40F0BB0E4EC3B29CFAF1D826791A]
2016-01-14 17:16:28,930 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:28,930 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791788418_0001_1][Event:VERTEX_FINISHED]: vertexName=E07E40F0BB0E4EC3B29CFAF1D826791A, vertexId=vertex_1452791788418_0001_1_01, initRequestedTime=1452791788595, initedTime=1452791788603, startRequestedTime=1452791788618, startedTime=1452791788618, finishTime=1452791788930, timeTaken=312, status=FAILED, diagnostics=Task failed, taskId=task_1452791788418_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1452791788418_0001_1_01 [E07E40F0BB0E4EC3B29CFAF1D826791A] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1452791788884, firstTasksToStart=[ task_1452791788418_0001_1_01_000000 ], lastTaskFinishTime=1452791788927, lastTasksToFinish=[ task_1452791788418_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[  ], longestDurationTasks=[  ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-01-14 17:16:28,930 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:28,930 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791788418_0001_1_01 [E07E40F0BB0E4EC3B29CFAF1D826791A] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-01-14 17:16:28,931 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791788418_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791788418_0001_00_000002, stoppedTime=1452791788931, exitStatus=0
2016-01-14 17:16:28,931 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:28,931 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791788418_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:28,932 INFO  node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1452791788418_0001_1_01_000000_0 failed: true container: container_1452791788418_0001_00_000002 numFailedTAs: 0
2016-01-14 17:16:28,932 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791788418_0001_1_01 [E07E40F0BB0E4EC3B29CFAF1D826791A] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-01-14 17:16:28,932 INFO  impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-01-14 17:16:28,932 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791788418_0001_1
2016-01-14 17:16:28,932 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1452791788418_0001_1_00 [C8192DFE78E04E2F93D37A91436CEB63]
2016-01-14 17:16:28,932 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 628E81455DFF41D98643A3D818C2057F for vertex: vertex_1452791788418_0001_1_01 [E07E40F0BB0E4EC3B29CFAF1D826791A]
2016-01-14 17:16:28,933 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=632815616, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791788919, Process_Duration=4, Process_End_Time=1452791788923, Read_Duration=0, Tuples_Read=1]]
2016-01-14 17:16:28,933 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 17:16:28,933 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 17:16:28,933 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E07E40F0BB0E4EC3B29CFAF1D826791A, mem on close (mb), free: 470, total: 603, max: 672
2016-01-14 17:16:28,934 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=E07E40F0BB0E4EC3B29CFAF1D826791A, index=1
2016-01-14 17:16:28,934 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791788418_0001_1, queueSize=0
2016-01-14 17:16:28,934 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:28,934 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: C8192DFE78E04E2F93D37A91436CEB63
2016-01-14 17:16:28,936 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:28,936 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:28,937 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:28,937 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:28,937 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:28,938 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=E07E40F0BB0E4EC3B29CFAF1D826791A, sourceVertex=C8192DFE78E04E2F93D37A91436CEB63
2016-01-14 17:16:28,938 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=E07E40F0BB0E4EC3B29CFAF1D826791A, sourceVertex=628E81455DFF41D98643A3D818C2057F
2016-01-14 17:16:28,938 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791788418_0001_00_000002
2016-01-14 17:16:28,938 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791788418_0001_00_000002
2016-01-14 17:16:28,939 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791788418_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-01-14 17:16:28,939 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1452791788418_0001_00_000002 completed but with errors
2016-01-14 17:16:28,939 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791788418_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791788418_0001_1, startTime=1452791788591, finishTime=1452791788932, timeTaken=341, status=FAILED, diagnostics=Vertex failed, vertexName=E07E40F0BB0E4EC3B29CFAF1D826791A, vertexId=vertex_1452791788418_0001_1_01, diagnostics=[Task failed, taskId=task_1452791788418_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1452791788418_0001_1_01 [E07E40F0BB0E4EC3B29CFAF1D826791A] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=179, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=632815616, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791788860, Process_Duration=4, Process_End_Time=1452791788864, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:28,944 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791788418_0001_1 finished with state: FAILED
2016-01-14 17:16:28,944 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791788418_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:28,944 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452791788418_0001_00_000002
2016-01-14 17:16:28,945 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1452791788418_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-01-14 17:16:28,945 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791788418_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:16:28 Completed Dag: dag_1452791788418_0001_1
2016-01-14 17:16:28,946 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791788418_0001_1, dagState=FAILED
2016-01-14 17:16:28,947 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791788418_0001_1_post
2016-01-14 17:16:28,947 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:28,947 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:28,947 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791788418_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-01-14 17:16:28,950 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...blyplatform/-1_VALUES_ALL, applicationId=application_1452791788418_0001
2016-01-14 17:16:28,953 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:28,953 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:28,953 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:28,953 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:28,953 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:28,955 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:28,955 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:28,956 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:28,956 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:28,957 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:28,957 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:28,957 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:28,958 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:28,962 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/031BE6876EC1423BA4E94963FC78E5FC/AB7FE986F66C4C22BDE9616793FD5944/.tez/application_1452791788418_0001
2016-01-14 17:16:28,962 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping all jobs
2016-01-14 17:16:28,963 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping: (1/1) ...blyplatform/-1_VALUES_ALL
2016-01-14 17:16:28,963 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopped all jobs
2016-01-14 17:16:28,964 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.567
2016-01-14 17:16:28,964 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_VALUES_ALL
2016-01-14 17:16:28,998 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:28,998 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:29,031 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.009
2016-01-14 17:16:29,040 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.018
2016-01-14 17:16:29,041 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:29,041 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:29,041 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:29,055 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:29,061 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_REPLACE
2016-01-14 17:16:29,105 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:29,105 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:29,106 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_REPLACE"]
2016-01-14 17:16:29,106 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:29,106 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:29,106 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:29,107 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...latform/-1_VALUES_REPLACE
2016-01-14 17:16:29,107 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:29,121 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:29,125 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:29,126 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:29,126 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:29,135 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/00DE3C6A358F45BA8FA1E7F1A2E0CA84/D4A2FF791EB04ECF97F885E60D150AF1/.tez/application_1452791789125_0001 doesn't exist and is created
2016-01-14 17:16:29,152 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:29,152 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:29,153 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/00DE3C6A358F45BA8FA1E7F1A2E0CA84/D4A2FF791EB04ECF97F885E60D150AF1/.tez/application_1452791789125_0001
2016-01-14 17:16:29,155 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791789125_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:29,165 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:29,166 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:29,167 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:29,167 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:29,168 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/00DE3C6A358F45BA8FA1E7F1A2E0CA84/D4A2FF791EB04ECF97F885E60D150AF1/.tez/application_1452791789125_0001/localmode-log-dir
2016-01-14 17:16:29,168 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/00DE3C6A358F45BA8FA1E7F1A2E0CA84/D4A2FF791EB04ECF97F885E60D150AF1/.tez/application_1452791789125_0001/localmode-log-dir/history.txt.appattempt_1452791789125_0001_000000, maxErrors=10
2016-01-14 17:16:29,168 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:29,168 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791789125_0001_000000, appSubmitTime=1452791789155, launchTime=1452791789155
2016-01-14 17:16:29,173 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:29,175 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:41049
2016-01-14 17:16:29,176 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:29,179 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:29,179 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:29,180 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:29,183 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791789125_0001_000000, startTime=1452791789183
2016-01-14 17:16:29,183 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:29,183 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:29,252 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:29,253 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:29,253 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/-1_VALUES_REPLACE, applicationId=application_1452791789125_0001, dagName=[00DE3C6A358F45BA8FA1E7F1A2E0CA84/D4A2FF791EB04ECF97F885E60D150AF1] every/(1/1) ...latform/-1_VALUES_REPLACE
2016-01-14 17:16:29,270 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [00DE3C6A358F45BA8FA1E7F1A2E0CA84/D4A2FF791EB04ECF97F885E60D150AF1] every/(1/1) ...latform/-1_VALUES_REPLACE
2016-01-14 17:16:29,271 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791789125_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/00DE3C6A358F45BA8FA1E7F1A2E0CA84/D4A2FF791EB04ECF97F885E60D150AF1/.tez/application_1452791789125_0001/localmode-log-dir/dag_1452791789125_0001_1.dot
2016-01-14 17:16:29,272 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/00DE3C6A358F45BA8FA1E7F1A2E0CA84/D4A2FF791EB04ECF97F885E60D150AF1/.tez/application_1452791789125_0001/localmode-log-dir/dag_1452791789125_0001_1-tez-dag.pb.txt
2016-01-14 17:16:29,282 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791789125_0001_1
2016-01-14 17:16:29,282 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:29,282 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [00DE3C6A358F45BA8FA1E7F1A2E0CA84/D4A2FF791EB04ECF97F885E60D150AF1] every/(1/1) ...latform/-1_VALUES_REPLACE
2016-01-14 17:16:29 Running Dag: dag_1452791789125_0001_1
2016-01-14 17:16:29,295 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:29,296 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791789125_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791789125_0001_1, submitTime=1452791789270
2016-01-14 17:16:29,296 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex BD86EA9EC0D740D6810BEBDCDB30DD43
2016-01-14 17:16:29,296 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:29,297 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:29,297 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791789125_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791789125_0001_1, initTime=1452791789296
2016-01-14 17:16:29,297 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791789125_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:29,298 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791789125_0001_1][Event:DAG_STARTED]: dagID=dag_1452791789125_0001_1, startTime=1452791789298
2016-01-14 17:16:29,300 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:29,300 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791789125_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:29,301 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: B2E95AA183C54457B67AE9FAE699AEE4 : {8336DF1C85E541CDBCB71FFA3C915456={InputName=8336DF1C85E541CDBCB71FFA3C915456}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:29,301 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 8336DF1C85E541CDBCB71FFA3C915456, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:29,301 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791789125_0001_1_00 [B2E95AA183C54457B67AE9FAE699AEE4]
2016-01-14 17:16:29,301 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791789125_0001_1_00 [B2E95AA183C54457B67AE9FAE699AEE4]
2016-01-14 17:16:29,298 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:29,301 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791789125_0001_1_00 [B2E95AA183C54457B67AE9FAE699AEE4]
2016-01-14 17:16:29,301 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791789125_0001_1_00 [B2E95AA183C54457B67AE9FAE699AEE4]. Starting root input initializers: 1
2016-01-14 17:16:29,301 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/-1_VALUES_REPLACE, applicationId=application_1452791789125_0001, dagName=[00DE3C6A358F45BA8FA1E7F1A2E0CA84/D4A2FF791EB04ECF97F885E60D150AF1] every/(1/1) ...latform/-1_VALUES_REPLACE
2016-01-14 17:16:29,302 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791789125_0001_1_00 [B2E95AA183C54457B67AE9FAE699AEE4] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:29,302 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 8336DF1C85E541CDBCB71FFA3C915456 on vertex vertex_1452791789125_0001_1_00 [B2E95AA183C54457B67AE9FAE699AEE4]
2016-01-14 17:16:29,302 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791789125_0001_1_01 [BD86EA9EC0D740D6810BEBDCDB30DD43]
2016-01-14 17:16:29,304 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:29,304 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 8336DF1C85E541CDBCB71FFA3C915456 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:29,304 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:29,305 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:29,305 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:29,306 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791789125_0001_1_01 [BD86EA9EC0D740D6810BEBDCDB30DD43]
2016-01-14 17:16:29,306 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791789125_0001_1_01 [BD86EA9EC0D740D6810BEBDCDB30DD43]
2016-01-14 17:16:29,306 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791789125_0001_1_01 [BD86EA9EC0D740D6810BEBDCDB30DD43]
2016-01-14 17:16:29,306 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=F95C2347BEB24AB388D604162409D839, vertexId=vertex_1452791789125_0001_1_01 [BD86EA9EC0D740D6810BEBDCDB30DD43], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:29,306 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:29,306 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 8336DF1C85E541CDBCB71FFA3C915456 on vertex vertex_1452791789125_0001_1_00 [B2E95AA183C54457B67AE9FAE699AEE4]
2016-01-14 17:16:29,306 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=F95C2347BEB24AB388D604162409D839, vertexId=vertex_1452791789125_0001_1_01 [BD86EA9EC0D740D6810BEBDCDB30DD43]
2016-01-14 17:16:29,309 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=F95C2347BEB24AB388D604162409D839, vertexName=BD86EA9EC0D740D6810BEBDCDB30DD43, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:29,309 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=F95C2347BEB24AB388D604162409D839, vertexId=vertex_1452791789125_0001_1_01 [BD86EA9EC0D740D6810BEBDCDB30DD43]
2016-01-14 17:16:29,310 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:29,311 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791789125_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BD86EA9EC0D740D6810BEBDCDB30DD43, vertexId=vertex_1452791789125_0001_1_01, initRequestedTime=1452791789302, initedTime=1452791789310, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:29,311 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791789125_0001_1_01 [BD86EA9EC0D740D6810BEBDCDB30DD43] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:29,311 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791789125_0001_1_00 [B2E95AA183C54457B67AE9FAE699AEE4] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:29,311 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {8336DF1C85E541CDBCB71FFA3C915456=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:29,312 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791789125_0001_1_00 [B2E95AA183C54457B67AE9FAE699AEE4] parallelism set to 1
2016-01-14 17:16:29,312 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791789125_0001_1_00 [B2E95AA183C54457B67AE9FAE699AEE4] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:29,312 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791789125_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791789125_0001_1_00, eventCount=1
2016-01-14 17:16:29,313 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:29,313 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791789125_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B2E95AA183C54457B67AE9FAE699AEE4, vertexId=vertex_1452791789125_0001_1_00, initRequestedTime=1452791789301, initedTime=1452791789313, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:29,314 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791789125_0001_1_00 [B2E95AA183C54457B67AE9FAE699AEE4] with distanceFromRoot: 0
2016-01-14 17:16:29,314 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791789125_0001_1_00 [B2E95AA183C54457B67AE9FAE699AEE4] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:29,314 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in B2E95AA183C54457B67AE9FAE699AEE4
2016-01-14 17:16:29,314 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791789125_0001_1_00 [B2E95AA183C54457B67AE9FAE699AEE4]
2016-01-14 17:16:29,315 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:29,315 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791789125_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791789125_0001_1_00, startRequestedTime=1452791789314, startedTime=1452791789314
2016-01-14 17:16:29,315 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791789125_0001_1_00 [B2E95AA183C54457B67AE9FAE699AEE4] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:29,315 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791789125_0001, with dag id: dag_1452791789125_0001_1
2016-01-14 17:16:29,316 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:29,316 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791789125_0001_1][Event:TASK_STARTED]: vertexName=B2E95AA183C54457B67AE9FAE699AEE4, taskId=task_1452791789125_0001_1_00_000000, scheduledTime=1452791789315, launchTime=1452791789315
2016-01-14 17:16:29,316 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791789125_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:29,317 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791789125_0001_1_00 for vertex: vertex_1452791789125_0001_1_01 [BD86EA9EC0D740D6810BEBDCDB30DD43] numStartedSources: 1 numSources: 1
2016-01-14 17:16:29,317 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791789125_0001_1_01 [BD86EA9EC0D740D6810BEBDCDB30DD43] with distanceFromRoot: 1
2016-01-14 17:16:29,317 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791789125_0001_1_00 [B2E95AA183C54457B67AE9FAE699AEE4]
2016-01-14 17:16:29,319 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: B2E95AA183C54457B67AE9FAE699AEE4 in vertex: BD86EA9EC0D740D6810BEBDCDB30DD43
2016-01-14 17:16:29,319 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: BD86EA9EC0D740D6810BEBDCDB30DD43 with 1 source tasks and 1 pending tasks
2016-01-14 17:16:29,320 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:29,321 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791789125_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791789125_0001_1_01, startRequestedTime=1452791789317, startedTime=1452791789317
2016-01-14 17:16:29,322 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791789125_0001_1_01 [BD86EA9EC0D740D6810BEBDCDB30DD43] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:29,322 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [00DE3C6A358F45BA8FA1E7F1A2E0CA84/D4A2FF791EB04ECF97F885E60D150AF1] every/(1/1) ...latform/-1_VALUES_REPLACE, VertexName: B2E95AA183C54457B67AE9FAE699AEE4, VertexParallelism: 1, TaskAttemptID:attempt_1452791789125_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=8336DF1C85E541CDBCB71FFA3C915456, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=BD86EA9EC0D740D6810BEBDCDB30DD43, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:29,322 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791789125_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:29,323 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:29,323 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:29,324 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:29,324 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:29,325 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791789125_0001_00_000001
2016-01-14 17:16:29,325 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791789125_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:29,326 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791789125_0001_00_000001
2016-01-14 17:16:29,326 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:29,327 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791789125_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:29,328 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:29,328 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791789125_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791789125_0001_00_000001, launchTime=1452791789328
2016-01-14 17:16:29,329 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791789125_0001_00_000001
2016-01-14 17:16:29,329 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791789125_0001_1_00_000000_0] to container: [container_1452791789125_0001_00_000001]
2016-01-14 17:16:29,329 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791789125_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:29,329 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791789125_0001_00_000001 given task: attempt_1452791789125_0001_1_00_000000_0
2016-01-14 17:16:29,330 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791789125_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791789125_0001_1_00_000000_0
2016-01-14 17:16:29,330 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791789125_0001_1_00_000000_0] started. Is using containerId: [container_1452791789125_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:29 Starting to run new task attempt: attempt_1452791789125_0001_1_00_000000_0
2016-01-14 17:16:29,330 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791789125_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B2E95AA183C54457B67AE9FAE699AEE4, taskAttemptId=attempt_1452791789125_0001_1_00_000000_0, startTime=1452791789330, containerId=container_1452791789125_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791789125_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:29,330 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791789125_0001_1_00_000000_0
2016-01-14 17:16:29,330 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791789125_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:29,330 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791789125_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:29,330 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:29,330 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:29,332 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:29,332 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:29,332 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:29,332 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:29,332 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:29,332 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [00DE3C6A358F45BA8FA1E7F1A2E0CA84/D4A2FF791EB04ECF97F885E60D150AF1] every/(1/1) ...latform/-1_VALUES_REPLACE, VertexName: B2E95AA183C54457B67AE9FAE699AEE4, VertexParallelism: 1, TaskAttemptID:attempt_1452791789125_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=8336DF1C85E541CDBCB71FFA3C915456, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=BD86EA9EC0D740D6810BEBDCDB30DD43, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:29,333 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:16:29,333 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791789125_0001_1_00_000000_0
2016-01-14 17:16:29,334 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:29,334 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:29,334 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791789125_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:29,335 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=8336DF1C85E541CDBCB71FFA3C915456, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:29,335 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=BD86EA9EC0D740D6810BEBDCDB30DD43, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:29,335 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:29,335 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:29,336 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:29,336 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: BD86EA9EC0D740D6810BEBDCDB30DD43
2016-01-14 17:16:29,338 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:29,338 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:29,339 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 8336DF1C85E541CDBCB71FFA3C915456
2016-01-14 17:16:29,339 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 8336DF1C85E541CDBCB71FFA3C915456
2016-01-14 17:16:29,340 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: BD86EA9EC0D740D6810BEBDCDB30DD43
2016-01-14 17:16:29,341 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: BD86EA9EC0D740D6810BEBDCDB30DD43
2016-01-14 17:16:29,342 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:29,342 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 8336DF1C85E541CDBCB71FFA3C915456
2016-01-14 17:16:29,343 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 8336DF1C85E541CDBCB71FFA3C915456
2016-01-14 17:16:29,345 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B2E95AA183C54457B67AE9FAE699AEE4, ordinal: 0
2016-01-14 17:16:29,345 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B2E95AA183C54457B67AE9FAE699AEE4, mem on start (mb), free: 396, total: 603, max: 672
2016-01-14 17:16:29,346 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:29,346 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:29,346 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:29,346 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:29,346 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:29,347 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:29,347 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:29,347 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:29,347 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:29,347 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 17:16:29,347 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 8336DF1C85E541CDBCB71FFA3C915456, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:29,348 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, BD86EA9EC0D740D6810BEBDCDB30DD43, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:29,348 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:29,348 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 8336DF1C85E541CDBCB71FFA3C915456 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:29,348 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:29,348 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:29,348 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 8336DF1C85E541CDBCB71FFA3C915456
2016-01-14 17:16:29,349 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:29,349 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 8336DF1C85E541CDBCB71FFA3C915456
2016-01-14 17:16:29,349 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:29,349 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791789125_0001_1_00_000000_0
2016-01-14 17:16:29,349 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:29,350 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@9bafdf2
2016-01-14 17:16:29,351 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@9bafdf2
2016-01-14 17:16:29,351 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:29,352 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:29,353 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:29,353 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 8336DF1C85E541CDBCB71FFA3C915456
2016-01-14 17:16:29,353 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 9359150BAC9F4B9E9377327EFEA060CD
2016-01-14 17:16:29,354 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 9359150BAC9F4B9E9377327EFEA060CD
2016-01-14 17:16:29,354 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@78e0e606; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@751dd1e9; comparator=cascading.tuple.hadoop.util.TupleComparator@78c05410; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:29,355 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:29,355 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:29,355 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:29,476 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:29,476 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:29,476 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:29,477 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:29,477 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:29,477 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:29,478 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:29,478 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B2E95AA183C54457B67AE9FAE699AEE4, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:29,481 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791789125_0001_1_00_000000_0
2016-01-14 17:16:29,481 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B2E95AA183C54457B67AE9FAE699AEE4, mem on close (mb), free: 602, total: 727, max: 727
2016-01-14 17:16:29,481 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:29,482 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:29,482 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:29,482 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:29,485 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:29 Completed running task attempt: attempt_1452791789125_0001_1_00_000000_0
2016-01-14 17:16:29,488 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791789125_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:29,489 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791789125_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791789125_0001_1_00, eventCount=1
2016-01-14 17:16:29,489 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=108, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=762839040, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791789478, Process_Duration=3, Process_End_Time=1452791789481, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:29,489 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:29,489 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:29,489 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:29,489 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791789125_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B2E95AA183C54457B67AE9FAE699AEE4, taskAttemptId=attempt_1452791789125_0001_1_00_000000_0, startTime=1452791789330, finishTime=1452791789489, timeTaken=159, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=108, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=762839040, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791789478, Process_Duration=3, Process_End_Time=1452791789481, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:29,490 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791789125_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:29,489 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791789125_0001_00_000001
2016-01-14 17:16:29,490 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791789125_0001_1_00_000000_0
2016-01-14 17:16:29,490 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791789125_0001_1][Event:TASK_FINISHED]: vertexName=B2E95AA183C54457B67AE9FAE699AEE4, taskId=task_1452791789125_0001_1_00_000000, startTime=1452791789330, finishTime=1452791789490, timeTaken=160, status=SUCCEEDED, successfulAttemptID=attempt_1452791789125_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=108, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=762839040, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791789478, Process_Duration=3, Process_End_Time=1452791789481, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:29,491 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791789125_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:29,491 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791789125_0001_1_00 [B2E95AA183C54457B67AE9FAE699AEE4] : 1
2016-01-14 17:16:29,491 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791789125_0001_1_00 [B2E95AA183C54457B67AE9FAE699AEE4]
2016-01-14 17:16:29,490 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:29,491 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:29,494 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791789125_0001_1][Event:VERTEX_FINISHED]: vertexName=B2E95AA183C54457B67AE9FAE699AEE4, vertexId=vertex_1452791789125_0001_1_00, initRequestedTime=1452791789301, initedTime=1452791789313, startRequestedTime=1452791789314, startedTime=1452791789314, finishTime=1452791789491, timeTaken=177, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=108, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=762839040, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791789478, Process_Duration=3, Process_End_Time=1452791789481, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791789330, firstTasksToStart=[ task_1452791789125_0001_1_00_000000 ], lastTaskFinishTime=1452791789489, lastTasksToFinish=[ task_1452791789125_0001_1_00_000000 ], minTaskDuration=159, maxTaskDuration=159, avgTaskDuration=159.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791789125_0001_1_00_000000 ], longestDurationTasks=[ task_1452791789125_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:29,494 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:29,495 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791789125_0001_1_00 [B2E95AA183C54457B67AE9FAE699AEE4] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:29,495 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791789125_0001_1_01 [BD86EA9EC0D740D6810BEBDCDB30DD43] attempt: attempt_1452791789125_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:29,495 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: BD86EA9EC0D740D6810BEBDCDB30DD43
2016-01-14 17:16:29,496 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791789125_0001_1_01 [BD86EA9EC0D740D6810BEBDCDB30DD43]
2016-01-14 17:16:29,496 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791789125_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:29,496 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791789125_0001_1_00 [B2E95AA183C54457B67AE9FAE699AEE4] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:29,497 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791789125_0001_1][Event:TASK_STARTED]: vertexName=BD86EA9EC0D740D6810BEBDCDB30DD43, taskId=task_1452791789125_0001_1_01_000000, scheduledTime=1452791789496, launchTime=1452791789496
2016-01-14 17:16:29,497 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791789125_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:29,497 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791789125_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791789125_0001_00_000001, stoppedTime=1452791789497, exitStatus=0
2016-01-14 17:16:29,498 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791789125_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:29,498 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791789125_0001_1_01 [BD86EA9EC0D740D6810BEBDCDB30DD43]
2016-01-14 17:16:29,497 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:29,498 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:29,499 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791789125_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:29,499 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [00DE3C6A358F45BA8FA1E7F1A2E0CA84/D4A2FF791EB04ECF97F885E60D150AF1] every/(1/1) ...latform/-1_VALUES_REPLACE, VertexName: BD86EA9EC0D740D6810BEBDCDB30DD43, VertexParallelism: 1, TaskAttemptID:attempt_1452791789125_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B2E95AA183C54457B67AE9FAE699AEE4, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=F95C2347BEB24AB388D604162409D839, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:29,499 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791789125_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:29,499 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791789125_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:29,499 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:29,500 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791789125_0001_00_000002
2016-01-14 17:16:29,500 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791789125_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:29,500 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791789125_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:29,501 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791789125_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:29,501 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791789125_0001_00_000001, Exiting
2016-01-14 17:16:29,501 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791789125_0001_00_000001
2016-01-14 17:16:29,501 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791789125_0001_00_000001
2016-01-14 17:16:29,502 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791789125_0001_00_000002
2016-01-14 17:16:29,502 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791789125_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:29,503 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791789125_0001_00_000001 completed successfully
2016-01-14 17:16:29,503 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 17:16:29,504 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791789125_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:29,505 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791789125_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:29,506 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:29,506 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791789125_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791789125_0001_00_000002, launchTime=1452791789505
2016-01-14 17:16:29,507 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791789125_0001_00_000002
2016-01-14 17:16:29,507 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791789125_0001_1_01_000000_0] to container: [container_1452791789125_0001_00_000002]
2016-01-14 17:16:29,507 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791789125_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:29,508 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791789125_0001_00_000002 given task: attempt_1452791789125_0001_1_01_000000_0
2016-01-14 17:16:29,508 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791789125_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791789125_0001_1_01_000000_0
2016-01-14 17:16:29,508 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791789125_0001_1_01_000000_0] started. Is using containerId: [container_1452791789125_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:29,508 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791789125_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BD86EA9EC0D740D6810BEBDCDB30DD43, taskAttemptId=attempt_1452791789125_0001_1_01_000000_0, startTime=1452791789508, containerId=container_1452791789125_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791789125_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:29,508 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791789125_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:29 Starting to run new task attempt: attempt_1452791789125_0001_1_01_000000_0
2016-01-14 17:16:29,508 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791789125_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:29,509 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791789125_0001_1_01_000000_0
2016-01-14 17:16:29,509 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:29,509 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:29,509 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:29,509 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:29,509 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:29,510 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:29,510 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [00DE3C6A358F45BA8FA1E7F1A2E0CA84/D4A2FF791EB04ECF97F885E60D150AF1] every/(1/1) ...latform/-1_VALUES_REPLACE, VertexName: BD86EA9EC0D740D6810BEBDCDB30DD43, VertexParallelism: 1, TaskAttemptID:attempt_1452791789125_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B2E95AA183C54457B67AE9FAE699AEE4, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=F95C2347BEB24AB388D604162409D839, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:29,510 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=762839040
2016-01-14 17:16:29,509 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:29,511 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791789125_0001_1_01_000000_0
2016-01-14 17:16:29,511 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:29,511 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:29,512 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B2E95AA183C54457B67AE9FAE699AEE4, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:29,512 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:29,512 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B2E95AA183C54457B67AE9FAE699AEE4
2016-01-14 17:16:29,513 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 686555136, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:29,513 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:29,513 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 686555136, type: INPUT, componentVertexName: B2E95AA183C54457B67AE9FAE699AEE4
2016-01-14 17:16:29,514 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B2E95AA183C54457B67AE9FAE699AEE4
2016-01-14 17:16:29,514 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:29,514 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791789125_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:29,516 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:29,517 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F95C2347BEB24AB388D604162409D839, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:29,517 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:29,517 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F95C2347BEB24AB388D604162409D839
2016-01-14 17:16:29,518 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:29,518 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: F95C2347BEB24AB388D604162409D839
2016-01-14 17:16:29,524 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BD86EA9EC0D740D6810BEBDCDB30DD43, ordinal: 1
2016-01-14 17:16:29,525 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BD86EA9EC0D740D6810BEBDCDB30DD43, mem on start (mb), free: 596, total: 727, max: 727
2016-01-14 17:16:29,525 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:29,525 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:29,525 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:29,529 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:29,529 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F95C2347BEB24AB388D604162409D839
2016-01-14 17:16:29,529 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:29,530 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:29,530 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:29,530 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:29,530 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:29,531 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:29,531 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 686555136, TotalRequestedScaled: 6.337432024615386E8, TotalJVMHeap: 762839040, TotalAvailable: 511102156, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:29,531 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B2E95AA183C54457B67AE9FAE699AEE4, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=686555136, allocated=511102156
2016-01-14 17:16:29,531 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F95C2347BEB24AB388D604162409D839, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:29,531 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:29,532 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B2E95AA183C54457B67AE9FAE699AEE4
2016-01-14 17:16:29,532 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B2E95AA183C54457B67AE9FAE699AEE4 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:29,532 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:29,533 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:29,534 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: B2E95AA183C54457B67AE9FAE699AEE4 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:29,534 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=686555136, postMergeMem=0, RuntimeTotalAvailable=511102156. Updated to: ShuffleMem=511102156, postMergeMem: 0
2016-01-14 17:16:29,534 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=511102156, maxSingleShuffleLimit=127775536, mergeThreshold=459991904, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:29,534 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:29,535 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:29,535 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B2E95AA183C54457B67AE9FAE699AEE4
2016-01-14 17:16:29,535 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:29,536 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:29,536 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791789125_0001_1_01_000000_0
2016-01-14 17:16:29,536 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791789125_0001_1_00_000000_0_10081, runDuration: 150316, ]
2016-01-14 17:16:29,538 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791789125_0001_1_00_000000_0_10081] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791789538, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 17:16:29,538 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : B2E95AA183C54457B67AE9FAE699AEE4
2016-01-14 17:16:29,538 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:29,539 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [B2E95AA183C54457B67AE9FAE699AEE4] #14 in 2ms
2016-01-14 17:16:29,539 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [B2E95AA183C54457B67AE9FAE699AEE4] #14, status:false, isInterrupted:false
2016-01-14 17:16:29,539 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:29,540 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:29,540 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:29,540 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:29,540 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:29,540 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 9359150BAC9F4B9E9377327EFEA060CD
2016-01-14 17:16:29,541 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_REPLACE"], id: F95C2347BEB24AB388D604162409D839
2016-01-14 17:16:29,541 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_REPLACE"]
2016-01-14 17:16:29,542 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 9359150BAC9F4B9E9377327EFEA060CD, for 1 inputs
2016-01-14 17:16:29,541 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:29,542 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : B2E95AA183C54457B67AE9FAE699AEE4
2016-01-14 17:16:29,542 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:29,543 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BD86EA9EC0D740D6810BEBDCDB30DD43, all 1 inputs ready in: 00:00:00.001
2016-01-14 17:16:29,543 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@2e50bfef; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@3768fbaa
2016-01-14 17:16:29,544 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:29,546 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(92)) - caught throwable
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:29,549 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:29,550 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:29,550 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791789125_0001_1_01_000000_0
2016-01-14 17:16:29,550 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791789125_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:29,551 INFO  task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1452791789125_0001_1_01_000000_0
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:29,556 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=528, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=762839040, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791789543, Process_Duration=8, Process_End_Time=1452791789551, Read_Duration=0, Tuples_Read=1]]
2016-01-14 17:16:29,557 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BD86EA9EC0D740D6810BEBDCDB30DD43, mem on close (mb), free: 593, total: 727, max: 727
2016-01-14 17:16:29,557 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=BD86EA9EC0D740D6810BEBDCDB30DD43, index=1
2016-01-14 17:16:29,556 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:29,556 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791789125_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BD86EA9EC0D740D6810BEBDCDB30DD43, taskAttemptId=attempt_1452791789125_0001_1_01_000000_0, startTime=1452791789508, finishTime=1452791789556, timeTaken=48, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=528, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=762839040, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791789543, Process_Duration=8, Process_End_Time=1452791789551, Read_Duration=0, Tuples_Read=1
2016-01-14 17:16:29,560 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791789125_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-01-14 17:16:29,560 INFO  impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1452791789125_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-01-14 17:16:29,561 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791789125_0001_1][Event:TASK_FINISHED]: vertexName=BD86EA9EC0D740D6810BEBDCDB30DD43, taskId=task_1452791789125_0001_1_01_000000, startTime=1452791789508, finishTime=1452791789561, timeTaken=53, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
], counters=Counters: 0
2016-01-14 17:16:29,561 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791789125_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-01-14 17:16:29,561 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791789125_0001_1_01 [BD86EA9EC0D740D6810BEBDCDB30DD43] : 1
2016-01-14 17:16:29,561 INFO  impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1452791789125_0001_1_01 [BD86EA9EC0D740D6810BEBDCDB30DD43] because task failed: task_1452791789125_0001_1_01_000000
2016-01-14 17:16:29,561 INFO  impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1452791789125_0001_1_01 [BD86EA9EC0D740D6810BEBDCDB30DD43] due to trigger: OWN_TASK_FAILURE
2016-01-14 17:16:29,561 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-01-14 17:16:29,562 INFO  impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1452791789125_0001_1_01 [BD86EA9EC0D740D6810BEBDCDB30DD43]
2016-01-14 17:16:29,562 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:29,562 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:29,558 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: B2E95AA183C54457B67AE9FAE699AEE4
2016-01-14 17:16:29,562 INFO  impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=F95C2347BEB24AB388D604162409D839, vertexId=vertex_1452791789125_0001_1_01 [BD86EA9EC0D740D6810BEBDCDB30DD43]
2016-01-14 17:16:29,565 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:29,566 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791789125_0001_1][Event:VERTEX_FINISHED]: vertexName=BD86EA9EC0D740D6810BEBDCDB30DD43, vertexId=vertex_1452791789125_0001_1_01, initRequestedTime=1452791789302, initedTime=1452791789310, startRequestedTime=1452791789317, startedTime=1452791789317, finishTime=1452791789564, timeTaken=247, status=FAILED, diagnostics=Task failed, taskId=task_1452791789125_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1452791789125_0001_1_01 [BD86EA9EC0D740D6810BEBDCDB30DD43] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1452791789508, firstTasksToStart=[ task_1452791789125_0001_1_01_000000 ], lastTaskFinishTime=1452791789556, lastTasksToFinish=[ task_1452791789125_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[  ], longestDurationTasks=[  ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-01-14 17:16:29,566 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791789125_0001_1_01 [BD86EA9EC0D740D6810BEBDCDB30DD43] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-01-14 17:16:29,566 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791789125_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791789125_0001_00_000002, stoppedTime=1452791789566, exitStatus=0
2016-01-14 17:16:29,566 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791789125_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:29,567 INFO  node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1452791789125_0001_1_01_000000_0 failed: true container: container_1452791789125_0001_00_000002 numFailedTAs: 0
2016-01-14 17:16:29,567 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791789125_0001_1_01 [BD86EA9EC0D740D6810BEBDCDB30DD43] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-01-14 17:16:29,567 INFO  impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-01-14 17:16:29,567 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791789125_0001_1
2016-01-14 17:16:29,567 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: F95C2347BEB24AB388D604162409D839 for vertex: vertex_1452791789125_0001_1_01 [BD86EA9EC0D740D6810BEBDCDB30DD43]
2016-01-14 17:16:29,567 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1452791789125_0001_1_00 [B2E95AA183C54457B67AE9FAE699AEE4]
2016-01-14 17:16:29,569 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:29,572 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791789125_0001_1, queueSize=0
2016-01-14 17:16:29,573 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:29,573 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791789125_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791789125_0001_1, startTime=1452791789298, finishTime=1452791789568, timeTaken=270, status=FAILED, diagnostics=Vertex failed, vertexName=BD86EA9EC0D740D6810BEBDCDB30DD43, vertexId=vertex_1452791789125_0001_1_01, diagnostics=[Task failed, taskId=task_1452791789125_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1452791789125_0001_1_01 [BD86EA9EC0D740D6810BEBDCDB30DD43] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=108, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=762839040, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791789478, Process_Duration=3, Process_End_Time=1452791789481, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:29,573 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791789125_0001_1 finished with state: FAILED
2016-01-14 17:16:29,574 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791789125_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:29,574 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791789125_0001_00_000002. Relying on regular task shutdown for it to end
2016-01-14 17:16:29,575 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:29,575 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:29,575 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:29,575 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:29,575 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:29,575 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=BD86EA9EC0D740D6810BEBDCDB30DD43, sourceVertex=B2E95AA183C54457B67AE9FAE699AEE4
2016-01-14 17:16:29,576 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=BD86EA9EC0D740D6810BEBDCDB30DD43, sourceVertex=F95C2347BEB24AB388D604162409D839
2016-01-14 17:16:29,576 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791789125_0001_00_000002
2016-01-14 17:16:29,576 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791789125_0001_00_000002
2016-01-14 17:16:29,577 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791789125_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-01-14 17:16:29,577 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1452791789125_0001_00_000002 completed but with errors
2016-01-14 17:16:29 Completed Dag: dag_1452791789125_0001_1
2016-01-14 17:16:29,577 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791789125_0001_1, dagState=FAILED
2016-01-14 17:16:29,578 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791789125_0001_1_post
2016-01-14 17:16:29,578 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:29,578 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:29,578 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791789125_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:29,578 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1452791789125_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-01-14 17:16:29,578 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791789125_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:29,578 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791789125_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-01-14 17:16:29,580 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/-1_VALUES_REPLACE, applicationId=application_1452791789125_0001
2016-01-14 17:16:29,581 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:29,581 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:29,582 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:29,582 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:29,582 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:29,582 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:29,582 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:29,583 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:29,583 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:29,583 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:29,583 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:29,584 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:29,584 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:29,588 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/00DE3C6A358F45BA8FA1E7F1A2E0CA84/D4A2FF791EB04ECF97F885E60D150AF1/.tez/application_1452791789125_0001
2016-01-14 17:16:29,588 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping all jobs
2016-01-14 17:16:29,588 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping: (1/1) ...latform/-1_VALUES_REPLACE
2016-01-14 17:16:29,589 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopped all jobs
2016-01-14 17:16:29,589 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.484
2016-01-14 17:16:29,589 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_VALUES_REPLACE
2016-01-14 17:16:29,623 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:29,623 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:29,645 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 17:16:29,645 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 17:16:29,662 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-01-14 17:16:29,669 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.021
2016-01-14 17:16:29,670 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:29,670 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:29,670 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:29,684 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:29,690 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_SWAP
2016-01-14 17:16:29,734 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:29,735 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:29,735 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_SWAP"]
2016-01-14 17:16:29,735 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:29,735 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:29,736 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:29,736 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...lyplatform/-1_VALUES_SWAP
2016-01-14 17:16:29,737 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:29,748 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:29,753 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:29,753 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:29,753 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:29,762 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1FC7E7B47B6545428F9A646F1D0A1073/E4301C4BAF404BA7863173BA44D10E00/.tez/application_1452791789753_0001 doesn't exist and is created
2016-01-14 17:16:29,778 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:29,778 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:29,778 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1FC7E7B47B6545428F9A646F1D0A1073/E4301C4BAF404BA7863173BA44D10E00/.tez/application_1452791789753_0001
2016-01-14 17:16:29,781 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791789753_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:29,792 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:29,792 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:29,793 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:29,794 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:29,794 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1FC7E7B47B6545428F9A646F1D0A1073/E4301C4BAF404BA7863173BA44D10E00/.tez/application_1452791789753_0001/localmode-log-dir
2016-01-14 17:16:29,794 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1FC7E7B47B6545428F9A646F1D0A1073/E4301C4BAF404BA7863173BA44D10E00/.tez/application_1452791789753_0001/localmode-log-dir/history.txt.appattempt_1452791789753_0001_000000, maxErrors=10
2016-01-14 17:16:29,794 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:29,794 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791789753_0001_000000, appSubmitTime=1452791789781, launchTime=1452791789781
2016-01-14 17:16:29,798 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:29,799 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:47217
2016-01-14 17:16:29,800 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:29,801 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:29,801 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:29,802 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:29,804 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791789753_0001_000000, startTime=1452791789804
2016-01-14 17:16:29,804 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:29,804 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:29,878 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:29,878 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:29,878 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...lyplatform/-1_VALUES_SWAP, applicationId=application_1452791789753_0001, dagName=[1FC7E7B47B6545428F9A646F1D0A1073/E4301C4BAF404BA7863173BA44D10E00] every/(1/1) ...lyplatform/-1_VALUES_SWAP
2016-01-14 17:16:29,894 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [1FC7E7B47B6545428F9A646F1D0A1073/E4301C4BAF404BA7863173BA44D10E00] every/(1/1) ...lyplatform/-1_VALUES_SWAP
2016-01-14 17:16:29,895 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791789753_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1FC7E7B47B6545428F9A646F1D0A1073/E4301C4BAF404BA7863173BA44D10E00/.tez/application_1452791789753_0001/localmode-log-dir/dag_1452791789753_0001_1.dot
2016-01-14 17:16:29,895 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1FC7E7B47B6545428F9A646F1D0A1073/E4301C4BAF404BA7863173BA44D10E00/.tez/application_1452791789753_0001/localmode-log-dir/dag_1452791789753_0001_1-tez-dag.pb.txt
2016-01-14 17:16:29,902 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791789753_0001_1
2016-01-14 17:16:29,902 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:29,902 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [1FC7E7B47B6545428F9A646F1D0A1073/E4301C4BAF404BA7863173BA44D10E00] every/(1/1) ...lyplatform/-1_VALUES_SWAP
2016-01-14 17:16:29 Running Dag: dag_1452791789753_0001_1
2016-01-14 17:16:29,914 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:29,914 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791789753_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791789753_0001_1, submitTime=1452791789894
2016-01-14 17:16:29,915 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex E8B57DF9B5BD46A7A51E7B6B2F82EF2B
2016-01-14 17:16:29,915 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:29,915 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:29,916 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791789753_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791789753_0001_1, initTime=1452791789914
2016-01-14 17:16:29,916 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791789753_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:29,916 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791789753_0001_1][Event:DAG_STARTED]: dagID=dag_1452791789753_0001_1, startTime=1452791789916
2016-01-14 17:16:29,916 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:29,917 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:29,917 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791789753_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:29,917 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: AA3DDAC45A9F4BD4978BD28AD46896FD : {828FD675E9994332A4A3A255E5EA0418={InputName=828FD675E9994332A4A3A255E5EA0418}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:29,917 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 828FD675E9994332A4A3A255E5EA0418, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:29,917 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791789753_0001_1_00 [AA3DDAC45A9F4BD4978BD28AD46896FD]
2016-01-14 17:16:29,918 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791789753_0001_1_00 [AA3DDAC45A9F4BD4978BD28AD46896FD]
2016-01-14 17:16:29,918 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791789753_0001_1_00 [AA3DDAC45A9F4BD4978BD28AD46896FD]
2016-01-14 17:16:29,918 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...lyplatform/-1_VALUES_SWAP, applicationId=application_1452791789753_0001, dagName=[1FC7E7B47B6545428F9A646F1D0A1073/E4301C4BAF404BA7863173BA44D10E00] every/(1/1) ...lyplatform/-1_VALUES_SWAP
2016-01-14 17:16:29,918 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791789753_0001_1_00 [AA3DDAC45A9F4BD4978BD28AD46896FD]. Starting root input initializers: 1
2016-01-14 17:16:29,919 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791789753_0001_1_00 [AA3DDAC45A9F4BD4978BD28AD46896FD] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:29,919 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791789753_0001_1_01 [E8B57DF9B5BD46A7A51E7B6B2F82EF2B]
2016-01-14 17:16:29,919 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 828FD675E9994332A4A3A255E5EA0418 on vertex vertex_1452791789753_0001_1_00 [AA3DDAC45A9F4BD4978BD28AD46896FD]
2016-01-14 17:16:29,921 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:29,921 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 828FD675E9994332A4A3A255E5EA0418 asking for 1 tasks. Headroom: 727 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:29,921 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:29,921 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:29,922 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:29,922 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791789753_0001_1_01 [E8B57DF9B5BD46A7A51E7B6B2F82EF2B]
2016-01-14 17:16:29,923 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791789753_0001_1_01 [E8B57DF9B5BD46A7A51E7B6B2F82EF2B]
2016-01-14 17:16:29,923 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:29,923 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791789753_0001_1_01 [E8B57DF9B5BD46A7A51E7B6B2F82EF2B]
2016-01-14 17:16:29,923 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=F085AB12A1CE48B9BD712BFDB59FB83B, vertexId=vertex_1452791789753_0001_1_01 [E8B57DF9B5BD46A7A51E7B6B2F82EF2B], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:29,923 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 828FD675E9994332A4A3A255E5EA0418 on vertex vertex_1452791789753_0001_1_00 [AA3DDAC45A9F4BD4978BD28AD46896FD]
2016-01-14 17:16:29,924 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=F085AB12A1CE48B9BD712BFDB59FB83B, vertexId=vertex_1452791789753_0001_1_01 [E8B57DF9B5BD46A7A51E7B6B2F82EF2B]
2016-01-14 17:16:29,926 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=F085AB12A1CE48B9BD712BFDB59FB83B, vertexName=E8B57DF9B5BD46A7A51E7B6B2F82EF2B, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:29,926 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=F085AB12A1CE48B9BD712BFDB59FB83B, vertexId=vertex_1452791789753_0001_1_01 [E8B57DF9B5BD46A7A51E7B6B2F82EF2B]
2016-01-14 17:16:29,927 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:29,927 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791789753_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E8B57DF9B5BD46A7A51E7B6B2F82EF2B, vertexId=vertex_1452791789753_0001_1_01, initRequestedTime=1452791789919, initedTime=1452791789927, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:29,928 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791789753_0001_1_01 [E8B57DF9B5BD46A7A51E7B6B2F82EF2B] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:29,928 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791789753_0001_1_00 [AA3DDAC45A9F4BD4978BD28AD46896FD] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:29,928 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {828FD675E9994332A4A3A255E5EA0418=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:29,929 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791789753_0001_1_00 [AA3DDAC45A9F4BD4978BD28AD46896FD] parallelism set to 1
2016-01-14 17:16:29,929 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791789753_0001_1_00 [AA3DDAC45A9F4BD4978BD28AD46896FD] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:29,929 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791789753_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791789753_0001_1_00, eventCount=1
2016-01-14 17:16:29,929 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791789753_0001_1][Event:VERTEX_INITIALIZED]: vertexName=AA3DDAC45A9F4BD4978BD28AD46896FD, vertexId=vertex_1452791789753_0001_1_00, initRequestedTime=1452791789917, initedTime=1452791789929, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:29,930 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791789753_0001_1_00 [AA3DDAC45A9F4BD4978BD28AD46896FD] with distanceFromRoot: 0
2016-01-14 17:16:29,930 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791789753_0001_1_00 [AA3DDAC45A9F4BD4978BD28AD46896FD] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:29,929 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:29,930 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in AA3DDAC45A9F4BD4978BD28AD46896FD
2016-01-14 17:16:29,931 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791789753_0001_1_00 [AA3DDAC45A9F4BD4978BD28AD46896FD]
2016-01-14 17:16:29,931 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791789753_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791789753_0001_1_00, startRequestedTime=1452791789930, startedTime=1452791789930
2016-01-14 17:16:29,931 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:29,931 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791789753_0001_1_00 [AA3DDAC45A9F4BD4978BD28AD46896FD] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:29,932 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:29,934 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791789753_0001, with dag id: dag_1452791789753_0001_1
2016-01-14 17:16:29,932 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791789753_0001_1][Event:TASK_STARTED]: vertexName=AA3DDAC45A9F4BD4978BD28AD46896FD, taskId=task_1452791789753_0001_1_00_000000, scheduledTime=1452791789932, launchTime=1452791789932
2016-01-14 17:16:29,934 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791789753_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:29,935 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791789753_0001_1_00 for vertex: vertex_1452791789753_0001_1_01 [E8B57DF9B5BD46A7A51E7B6B2F82EF2B] numStartedSources: 1 numSources: 1
2016-01-14 17:16:29,935 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791789753_0001_1_01 [E8B57DF9B5BD46A7A51E7B6B2F82EF2B] with distanceFromRoot: 1
2016-01-14 17:16:29,935 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791789753_0001_1_00 [AA3DDAC45A9F4BD4978BD28AD46896FD]
2016-01-14 17:16:29,936 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: AA3DDAC45A9F4BD4978BD28AD46896FD in vertex: E8B57DF9B5BD46A7A51E7B6B2F82EF2B
2016-01-14 17:16:29,936 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: E8B57DF9B5BD46A7A51E7B6B2F82EF2B with 1 source tasks and 1 pending tasks
2016-01-14 17:16:29,937 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:29,937 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791789753_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791789753_0001_1_01, startRequestedTime=1452791789935, startedTime=1452791789935
2016-01-14 17:16:29,937 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791789753_0001_1_01 [E8B57DF9B5BD46A7A51E7B6B2F82EF2B] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:29,938 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [1FC7E7B47B6545428F9A646F1D0A1073/E4301C4BAF404BA7863173BA44D10E00] every/(1/1) ...lyplatform/-1_VALUES_SWAP, VertexName: AA3DDAC45A9F4BD4978BD28AD46896FD, VertexParallelism: 1, TaskAttemptID:attempt_1452791789753_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=828FD675E9994332A4A3A255E5EA0418, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E8B57DF9B5BD46A7A51E7B6B2F82EF2B, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:29,939 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791789753_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:29,940 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:29,941 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:29,941 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:29,941 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:29,942 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791789753_0001_00_000001
2016-01-14 17:16:29,942 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791789753_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:29,943 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791789753_0001_00_000001
2016-01-14 17:16:29,944 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:29,944 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791789753_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:29,944 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:29,945 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791789753_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791789753_0001_00_000001, launchTime=1452791789944
2016-01-14 17:16:29,945 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791789753_0001_00_000001
2016-01-14 17:16:29,946 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791789753_0001_1_00_000000_0] to container: [container_1452791789753_0001_00_000001]
2016-01-14 17:16:29,946 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791789753_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:29,946 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791789753_0001_00_000001 given task: attempt_1452791789753_0001_1_00_000000_0
2016-01-14 17:16:29,946 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791789753_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791789753_0001_1_00_000000_0
2016-01-14 17:16:29,946 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791789753_0001_1_00_000000_0] started. Is using containerId: [container_1452791789753_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:29,947 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791789753_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=AA3DDAC45A9F4BD4978BD28AD46896FD, taskAttemptId=attempt_1452791789753_0001_1_00_000000_0, startTime=1452791789946, containerId=container_1452791789753_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791789753_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:29,947 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791789753_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:29,947 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791789753_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:29,947 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:29 Starting to run new task attempt: attempt_1452791789753_0001_1_00_000000_0
2016-01-14 17:16:29,948 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791789753_0001_1_00_000000_0
2016-01-14 17:16:29,948 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:29,948 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:29,948 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:29,948 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:29,948 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:29,949 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:29,949 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [1FC7E7B47B6545428F9A646F1D0A1073/E4301C4BAF404BA7863173BA44D10E00] every/(1/1) ...lyplatform/-1_VALUES_SWAP, VertexName: AA3DDAC45A9F4BD4978BD28AD46896FD, VertexParallelism: 1, TaskAttemptID:attempt_1452791789753_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=828FD675E9994332A4A3A255E5EA0418, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E8B57DF9B5BD46A7A51E7B6B2F82EF2B, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:29,949 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=762839040
2016-01-14 17:16:29,950 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791789753_0001_1_00_000000_0
2016-01-14 17:16:29,950 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:29,950 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:29,950 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=828FD675E9994332A4A3A255E5EA0418, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:29,950 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:29,951 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=E8B57DF9B5BD46A7A51E7B6B2F82EF2B, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:29,952 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:29,952 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: E8B57DF9B5BD46A7A51E7B6B2F82EF2B
2016-01-14 17:16:29,953 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:29,954 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:29,954 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 828FD675E9994332A4A3A255E5EA0418
2016-01-14 17:16:29,954 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 828FD675E9994332A4A3A255E5EA0418
2016-01-14 17:16:29,955 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:29,955 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: E8B57DF9B5BD46A7A51E7B6B2F82EF2B
2016-01-14 17:16:29,959 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: E8B57DF9B5BD46A7A51E7B6B2F82EF2B
2016-01-14 17:16:29,960 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:29,960 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 828FD675E9994332A4A3A255E5EA0418
2016-01-14 17:16:29,960 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 828FD675E9994332A4A3A255E5EA0418
2016-01-14 17:16:29,961 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791789753_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:29,963 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: AA3DDAC45A9F4BD4978BD28AD46896FD, ordinal: 0
2016-01-14 17:16:29,963 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AA3DDAC45A9F4BD4978BD28AD46896FD, mem on start (mb), free: 518, total: 727, max: 727
2016-01-14 17:16:29,963 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:29,963 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:29,963 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:29,963 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:29,963 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:29,964 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:29,964 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:29,964 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:29,964 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:29,964 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 762839040, TotalAvailable: 511102156, TotalRequested/TotalJVMHeap:0.14
2016-01-14 17:16:29,964 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 828FD675E9994332A4A3A255E5EA0418, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:29,965 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, E8B57DF9B5BD46A7A51E7B6B2F82EF2B, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:29,965 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:29,965 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 828FD675E9994332A4A3A255E5EA0418 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:29,965 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:29,966 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:29,965 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 828FD675E9994332A4A3A255E5EA0418
2016-01-14 17:16:29,966 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:29,967 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 828FD675E9994332A4A3A255E5EA0418
2016-01-14 17:16:29,967 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:29,967 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791789753_0001_1_00_000000_0
2016-01-14 17:16:29,968 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:29,968 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@6471dda2
2016-01-14 17:16:29,971 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@6471dda2
2016-01-14 17:16:29,971 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:29,971 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:29,971 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:29,972 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 828FD675E9994332A4A3A255E5EA0418
2016-01-14 17:16:29,972 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: A88DBAE49A8B4C798760969DE8D17EC5
2016-01-14 17:16:29,972 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] A88DBAE49A8B4C798760969DE8D17EC5
2016-01-14 17:16:29,975 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@427f1590; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@2560c30b; comparator=cascading.tuple.hadoop.util.TupleComparator@73a51a58; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:29,976 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:29,976 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:29,976 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:30,049 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:30,049 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:30,049 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:30,049 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:30,049 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:30,051 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:30,051 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:30,051 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: AA3DDAC45A9F4BD4978BD28AD46896FD, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:30,055 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791789753_0001_1_00_000000_0
2016-01-14 17:16:30,055 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AA3DDAC45A9F4BD4978BD28AD46896FD, mem on close (mb), free: 398, total: 728, max: 728
2016-01-14 17:16:30,055 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:30,055 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:30,055 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:30,055 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:30,059 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:30,061 WARN  task.TaskReporter (TaskReporter.java:call(199)) - Exiting TaskReporter thread with pending queue size=2
2016-01-14 17:16:30 Completed running task attempt: attempt_1452791789753_0001_1_00_000000_0
2016-01-14 17:16:30,062 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791789753_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:30,062 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791789753_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791789753_0001_1_00, eventCount=1
2016-01-14 17:16:30,063 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:30,063 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791789753_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=AA3DDAC45A9F4BD4978BD28AD46896FD, taskAttemptId=attempt_1452791789753_0001_1_00_000000_0, startTime=1452791789946, finishTime=1452791790063, timeTaken=117, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=60, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=763363328, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791790051, Process_Duration=4, Process_End_Time=1452791790055, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:30,063 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791789753_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:30,063 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:30,063 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791789753_0001_1_00_000000_0
2016-01-14 17:16:30,064 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:30,064 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791789753_0001_1][Event:TASK_FINISHED]: vertexName=AA3DDAC45A9F4BD4978BD28AD46896FD, taskId=task_1452791789753_0001_1_00_000000, startTime=1452791789946, finishTime=1452791790064, timeTaken=118, status=SUCCEEDED, successfulAttemptID=attempt_1452791789753_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=60, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=763363328, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791790051, Process_Duration=4, Process_End_Time=1452791790055, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:30,064 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791789753_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:30,065 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791789753_0001_1_00 [AA3DDAC45A9F4BD4978BD28AD46896FD] : 1
2016-01-14 17:16:30,065 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791789753_0001_1_00 [AA3DDAC45A9F4BD4978BD28AD46896FD]
2016-01-14 17:16:30,065 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=60, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=763363328, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791790051, Process_Duration=4, Process_End_Time=1452791790055, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=2][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:30,065 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:30,065 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:30,066 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791789753_0001_00_000001
2016-01-14 17:16:30,066 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791789753_0001_1][Event:VERTEX_FINISHED]: vertexName=AA3DDAC45A9F4BD4978BD28AD46896FD, vertexId=vertex_1452791789753_0001_1_00, initRequestedTime=1452791789917, initedTime=1452791789929, startRequestedTime=1452791789930, startedTime=1452791789930, finishTime=1452791790065, timeTaken=135, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=60, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=763363328, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791790051, Process_Duration=4, Process_End_Time=1452791790055, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791789946, firstTasksToStart=[ task_1452791789753_0001_1_00_000000 ], lastTaskFinishTime=1452791790063, lastTasksToFinish=[ task_1452791789753_0001_1_00_000000 ], minTaskDuration=117, maxTaskDuration=117, avgTaskDuration=117.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791789753_0001_1_00_000000 ], longestDurationTasks=[ task_1452791789753_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:30,066 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:30,066 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791789753_0001_1_00 [AA3DDAC45A9F4BD4978BD28AD46896FD] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:30,066 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791789753_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:30,067 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791789753_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791789753_0001_00_000001, stoppedTime=1452791790067, exitStatus=0
2016-01-14 17:16:30,067 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791789753_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:30,067 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:30,067 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791789753_0001_1_01 [E8B57DF9B5BD46A7A51E7B6B2F82EF2B] attempt: attempt_1452791789753_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:30,067 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: E8B57DF9B5BD46A7A51E7B6B2F82EF2B
2016-01-14 17:16:30,067 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791789753_0001_1_01 [E8B57DF9B5BD46A7A51E7B6B2F82EF2B]
2016-01-14 17:16:30,067 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791789753_0001_1_00 [AA3DDAC45A9F4BD4978BD28AD46896FD] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:30,068 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791789753_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:30,068 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791789753_0001_1][Event:TASK_STARTED]: vertexName=E8B57DF9B5BD46A7A51E7B6B2F82EF2B, taskId=task_1452791789753_0001_1_01_000000, scheduledTime=1452791790068, launchTime=1452791790068
2016-01-14 17:16:30,068 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791789753_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:30,068 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791789753_0001_1_01 [E8B57DF9B5BD46A7A51E7B6B2F82EF2B]
2016-01-14 17:16:30,068 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791789753_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:30,068 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [1FC7E7B47B6545428F9A646F1D0A1073/E4301C4BAF404BA7863173BA44D10E00] every/(1/1) ...lyplatform/-1_VALUES_SWAP, VertexName: E8B57DF9B5BD46A7A51E7B6B2F82EF2B, VertexParallelism: 1, TaskAttemptID:attempt_1452791789753_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=AA3DDAC45A9F4BD4978BD28AD46896FD, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=F085AB12A1CE48B9BD712BFDB59FB83B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:30,068 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791789753_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:30,069 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:30,069 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:30,069 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791789753_0001_00_000002
2016-01-14 17:16:30,070 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791789753_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:30,071 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791789753_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:30,071 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791789753_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:30,071 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791789753_0001_00_000002
2016-01-14 17:16:30,072 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:30,071 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791789753_0001_00_000001, Exiting
2016-01-14 17:16:30,072 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791789753_0001_00_000001
2016-01-14 17:16:30,072 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791789753_0001_00_000001
2016-01-14 17:16:30,073 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791789753_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:30,073 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:30,073 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791789753_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:30,073 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791789753_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791789753_0001_00_000002, launchTime=1452791790073
2016-01-14 17:16:30,073 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791789753_0001_00_000001 completed successfully
2016-01-14 17:16:30,074 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791789753_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:30,074 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791789753_0001_00_000002
2016-01-14 17:16:30,075 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791789753_0001_1_01_000000_0] to container: [container_1452791789753_0001_00_000002]
2016-01-14 17:16:30,075 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791789753_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:30,075 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791789753_0001_00_000002 given task: attempt_1452791789753_0001_1_01_000000_0
2016-01-14 17:16:30,075 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791789753_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791789753_0001_1_01_000000_0
2016-01-14 17:16:30,075 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791789753_0001_1_01_000000_0] started. Is using containerId: [container_1452791789753_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:30,076 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791789753_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E8B57DF9B5BD46A7A51E7B6B2F82EF2B, taskAttemptId=attempt_1452791789753_0001_1_01_000000_0, startTime=1452791790075, containerId=container_1452791789753_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791789753_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:30,076 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791789753_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:30,076 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791789753_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:30 Starting to run new task attempt: attempt_1452791789753_0001_1_01_000000_0
2016-01-14 17:16:30,076 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791789753_0001_1_01_000000_0
2016-01-14 17:16:30,076 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:30,076 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:30,076 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:30,077 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:30,077 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:30,077 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:30,077 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:30,077 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [1FC7E7B47B6545428F9A646F1D0A1073/E4301C4BAF404BA7863173BA44D10E00] every/(1/1) ...lyplatform/-1_VALUES_SWAP, VertexName: E8B57DF9B5BD46A7A51E7B6B2F82EF2B, VertexParallelism: 1, TaskAttemptID:attempt_1452791789753_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=AA3DDAC45A9F4BD4978BD28AD46896FD, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=F085AB12A1CE48B9BD712BFDB59FB83B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:30,077 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=763363328
2016-01-14 17:16:30,078 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791789753_0001_1_01_000000_0
2016-01-14 17:16:30,078 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:30,078 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:30,078 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:30,081 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:30,082 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F085AB12A1CE48B9BD712BFDB59FB83B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:30,082 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:30,083 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F085AB12A1CE48B9BD712BFDB59FB83B
2016-01-14 17:16:30,083 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:30,083 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: F085AB12A1CE48B9BD712BFDB59FB83B
2016-01-14 17:16:30,082 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=AA3DDAC45A9F4BD4978BD28AD46896FD, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:30,084 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:30,085 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: AA3DDAC45A9F4BD4978BD28AD46896FD
2016-01-14 17:16:30,086 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 687027008, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:30,087 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:30,087 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 687027008, type: INPUT, componentVertexName: AA3DDAC45A9F4BD4978BD28AD46896FD
2016-01-14 17:16:30,088 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: AA3DDAC45A9F4BD4978BD28AD46896FD
2016-01-14 17:16:30,088 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791789753_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:30,090 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E8B57DF9B5BD46A7A51E7B6B2F82EF2B, ordinal: 1
2016-01-14 17:16:30,090 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E8B57DF9B5BD46A7A51E7B6B2F82EF2B, mem on start (mb), free: 392, total: 728, max: 728
2016-01-14 17:16:30,090 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:30,090 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:30,090 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:30,094 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:30,094 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F085AB12A1CE48B9BD712BFDB59FB83B
2016-01-14 17:16:30,094 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:30,094 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:30,095 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:30,095 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:30,095 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:30,095 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:30,095 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 687027008, TotalRequestedScaled: 6.341787766153847E8, TotalJVMHeap: 763363328, TotalAvailable: 511453429, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:30,096 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F085AB12A1CE48B9BD712BFDB59FB83B, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:30,096 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, AA3DDAC45A9F4BD4978BD28AD46896FD, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=687027008, allocated=511453429
2016-01-14 17:16:30,096 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:30,096 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: AA3DDAC45A9F4BD4978BD28AD46896FD being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:30,096 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: AA3DDAC45A9F4BD4978BD28AD46896FD
2016-01-14 17:16:30,097 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:30,097 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:30,097 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:30,097 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:30,098 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: AA3DDAC45A9F4BD4978BD28AD46896FD with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:30,098 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=687027008, postMergeMem=0, RuntimeTotalAvailable=511453429. Updated to: ShuffleMem=511453429, postMergeMem: 0
2016-01-14 17:16:30,098 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=511453429, maxSingleShuffleLimit=127863360, mergeThreshold=460308096, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:30,099 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:30,101 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: AA3DDAC45A9F4BD4978BD28AD46896FD
2016-01-14 17:16:30,101 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:30,101 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791789753_0001_1_01_000000_0
2016-01-14 17:16:30,103 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791789753_0001_1_00_000000_0_10086, runDuration: 107415, ]
2016-01-14 17:16:30,104 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791789753_0001_1_00_000000_0_10086] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791790104, TimeTaken=0, Rate=0.00 MB/s
2016-01-14 17:16:30,105 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : AA3DDAC45A9F4BD4978BD28AD46896FD
2016-01-14 17:16:30,105 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:30,105 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [AA3DDAC45A9F4BD4978BD28AD46896FD] #15 in 1ms
2016-01-14 17:16:30,106 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:30,106 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [AA3DDAC45A9F4BD4978BD28AD46896FD] #15, status:false, isInterrupted:false
2016-01-14 17:16:30,107 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:30,107 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: A88DBAE49A8B4C798760969DE8D17EC5
2016-01-14 17:16:30,107 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:30,107 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_SWAP"], id: F085AB12A1CE48B9BD712BFDB59FB83B
2016-01-14 17:16:30,108 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:30,108 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:30,108 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:30,108 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : AA3DDAC45A9F4BD4978BD28AD46896FD
2016-01-14 17:16:30,108 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:30,108 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_SWAP"]
2016-01-14 17:16:30,109 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] A88DBAE49A8B4C798760969DE8D17EC5, for 1 inputs
2016-01-14 17:16:30,109 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E8B57DF9B5BD46A7A51E7B6B2F82EF2B, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:30,109 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@65b17bf8; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@67420af0
2016-01-14 17:16:30,110 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:30,118 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(92)) - caught throwable
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:30,125 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:30,125 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:30,126 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791789753_0001_1_01_000000_0
2016-01-14 17:16:30,126 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791789753_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:30,127 INFO  task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1452791789753_0001_1_01_000000_0
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:30,134 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=528, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=763363328, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791790109, Process_Duration=18, Process_End_Time=1452791790127, Read_Duration=0, Tuples_Read=1]]
2016-01-14 17:16:30,135 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791789753_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E8B57DF9B5BD46A7A51E7B6B2F82EF2B, taskAttemptId=attempt_1452791789753_0001_1_01_000000_0, startTime=1452791790075, finishTime=1452791790134, timeTaken=59, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=528, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=763363328, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791790109, Process_Duration=18, Process_End_Time=1452791790127, Read_Duration=0, Tuples_Read=1
2016-01-14 17:16:30,135 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:30,135 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791789753_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-01-14 17:16:30,135 INFO  impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1452791789753_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-01-14 17:16:30,135 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791789753_0001_1][Event:TASK_FINISHED]: vertexName=E8B57DF9B5BD46A7A51E7B6B2F82EF2B, taskId=task_1452791789753_0001_1_01_000000, startTime=1452791790075, finishTime=1452791790135, timeTaken=60, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
], counters=Counters: 0
2016-01-14 17:16:30,136 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:30,135 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E8B57DF9B5BD46A7A51E7B6B2F82EF2B, mem on close (mb), free: 388, total: 728, max: 728
2016-01-14 17:16:30,136 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=E8B57DF9B5BD46A7A51E7B6B2F82EF2B, index=1
2016-01-14 17:16:30,136 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791789753_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-01-14 17:16:30,137 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: AA3DDAC45A9F4BD4978BD28AD46896FD
2016-01-14 17:16:30,137 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791789753_0001_1_01 [E8B57DF9B5BD46A7A51E7B6B2F82EF2B] : 1
2016-01-14 17:16:30,137 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:30,137 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:30,137 INFO  impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1452791789753_0001_1_01 [E8B57DF9B5BD46A7A51E7B6B2F82EF2B] because task failed: task_1452791789753_0001_1_01_000000
2016-01-14 17:16:30,137 INFO  impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1452791789753_0001_1_01 [E8B57DF9B5BD46A7A51E7B6B2F82EF2B] due to trigger: OWN_TASK_FAILURE
2016-01-14 17:16:30,137 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-01-14 17:16:30,138 INFO  impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1452791789753_0001_1_01 [E8B57DF9B5BD46A7A51E7B6B2F82EF2B]
2016-01-14 17:16:30,138 INFO  impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=F085AB12A1CE48B9BD712BFDB59FB83B, vertexId=vertex_1452791789753_0001_1_01 [E8B57DF9B5BD46A7A51E7B6B2F82EF2B]
2016-01-14 17:16:30,138 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:30,138 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:30,139 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:30,139 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:30,139 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=E8B57DF9B5BD46A7A51E7B6B2F82EF2B, sourceVertex=AA3DDAC45A9F4BD4978BD28AD46896FD
2016-01-14 17:16:30,139 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:30,139 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=E8B57DF9B5BD46A7A51E7B6B2F82EF2B, sourceVertex=F085AB12A1CE48B9BD712BFDB59FB83B
2016-01-14 17:16:30,140 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791789753_0001_1][Event:VERTEX_FINISHED]: vertexName=E8B57DF9B5BD46A7A51E7B6B2F82EF2B, vertexId=vertex_1452791789753_0001_1_01, initRequestedTime=1452791789919, initedTime=1452791789927, startRequestedTime=1452791789935, startedTime=1452791789935, finishTime=1452791790139, timeTaken=204, status=FAILED, diagnostics=Task failed, taskId=task_1452791789753_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1452791789753_0001_1_01 [E8B57DF9B5BD46A7A51E7B6B2F82EF2B] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1452791790075, firstTasksToStart=[ task_1452791789753_0001_1_01_000000 ], lastTaskFinishTime=1452791790134, lastTasksToFinish=[ task_1452791789753_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[  ], longestDurationTasks=[  ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-01-14 17:16:30,140 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791789753_0001_00_000002
2016-01-14 17:16:30,140 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791789753_0001_00_000002
2016-01-14 17:16:30,140 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791789753_0001_1_01 [E8B57DF9B5BD46A7A51E7B6B2F82EF2B] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-01-14 17:16:30,141 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791789753_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-01-14 17:16:30,141 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:30,141 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1452791789753_0001_00_000002 completed but with errors
2016-01-14 17:16:30,141 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791789753_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791789753_0001_00_000002, stoppedTime=1452791790141, exitStatus=0
2016-01-14 17:16:30,141 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791789753_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:30,141 INFO  node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1452791789753_0001_1_01_000000_0 failed: true container: container_1452791789753_0001_00_000002 numFailedTAs: 0
2016-01-14 17:16:30,141 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791789753_0001_1_01 [E8B57DF9B5BD46A7A51E7B6B2F82EF2B] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-01-14 17:16:30,142 INFO  impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-01-14 17:16:30,142 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791789753_0001_1
2016-01-14 17:16:30,142 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1452791789753_0001_1_00 [AA3DDAC45A9F4BD4978BD28AD46896FD]
2016-01-14 17:16:30,142 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: F085AB12A1CE48B9BD712BFDB59FB83B for vertex: vertex_1452791789753_0001_1_01 [E8B57DF9B5BD46A7A51E7B6B2F82EF2B]
2016-01-14 17:16:30,145 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791789753_0001_1, queueSize=0
2016-01-14 17:16:30,145 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:30,146 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791789753_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791789753_0001_1, startTime=1452791789916, finishTime=1452791790142, timeTaken=226, status=FAILED, diagnostics=Vertex failed, vertexName=E8B57DF9B5BD46A7A51E7B6B2F82EF2B, vertexId=vertex_1452791789753_0001_1_01, diagnostics=[Task failed, taskId=task_1452791789753_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1452791789753_0001_1_01 [E8B57DF9B5BD46A7A51E7B6B2F82EF2B] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=60, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=763363328, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791790051, Process_Duration=4, Process_End_Time=1452791790055, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:30,146 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791789753_0001_1 finished with state: FAILED
2016-01-14 17:16:30,146 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791789753_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:30,147 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1452791789753_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-01-14 17:16:30,147 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791789753_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:16:30,147 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452791789753_0001_00_000002
2016-01-14 17:16:30 Completed Dag: dag_1452791789753_0001_1
2016-01-14 17:16:30,148 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791789753_0001_1, dagState=FAILED
2016-01-14 17:16:30,148 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791789753_0001_1_post
2016-01-14 17:16:30,148 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:30,148 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:30,149 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791789753_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-01-14 17:16:30,152 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...lyplatform/-1_VALUES_SWAP, applicationId=application_1452791789753_0001
2016-01-14 17:16:30,153 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:30,153 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:30,153 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:30,153 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:30,154 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:30,154 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:30,154 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:30,154 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:30,155 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:30,155 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:30,155 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:30,156 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:30,156 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:30,162 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1FC7E7B47B6545428F9A646F1D0A1073/E4301C4BAF404BA7863173BA44D10E00/.tez/application_1452791789753_0001
2016-01-14 17:16:30,162 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping all jobs
2016-01-14 17:16:30,163 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping: (1/1) ...lyplatform/-1_VALUES_SWAP
2016-01-14 17:16:30,163 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopped all jobs
2016-01-14 17:16:30,164 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.429
2016-01-14 17:16:30,164 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_VALUES_SWAP
2016-01-14 17:16:30,187 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:30,187 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:30,220 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-01-14 17:16:30,229 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.018
2016-01-14 17:16:30,229 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:30,229 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:30,229 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:30,244 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:30,253 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_ARGS_-1
2016-01-14 17:16:30,300 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:30,300 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:30,300 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_ARGS_-1"]
2016-01-14 17:16:30,300 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:30,300 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:30,300 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:30,303 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...semblyplatform/-1_ARGS_-1
2016-01-14 17:16:30,303 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:30,316 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 17:16:30,316 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 17:16:30,316 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:30,321 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:30,322 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:30,322 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:30,333 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/B048CDD5A8E84A8995BDB2B2311D0BCC/DD493CA5713E4EF4BC10680AB04709A0/.tez/application_1452791790322_0001 doesn't exist and is created
2016-01-14 17:16:30,350 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:30,351 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:30,352 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/B048CDD5A8E84A8995BDB2B2311D0BCC/DD493CA5713E4EF4BC10680AB04709A0/.tez/application_1452791790322_0001
2016-01-14 17:16:30,355 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791790322_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:30,366 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:30,367 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:30,369 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:30,369 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:30,370 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/B048CDD5A8E84A8995BDB2B2311D0BCC/DD493CA5713E4EF4BC10680AB04709A0/.tez/application_1452791790322_0001/localmode-log-dir
2016-01-14 17:16:30,371 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/B048CDD5A8E84A8995BDB2B2311D0BCC/DD493CA5713E4EF4BC10680AB04709A0/.tez/application_1452791790322_0001/localmode-log-dir/history.txt.appattempt_1452791790322_0001_000000, maxErrors=10
2016-01-14 17:16:30,371 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:30,372 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791790322_0001_000000, appSubmitTime=1452791790354, launchTime=1452791790355
2016-01-14 17:16:30,375 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:30,377 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:30,378 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:30,379 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:30,386 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:56559
2016-01-14 17:16:30,388 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:30,392 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791790322_0001_000000, startTime=1452791790392
2016-01-14 17:16:30,392 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:30,392 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:30,451 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:30,451 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:30,451 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...semblyplatform/-1_ARGS_-1, applicationId=application_1452791790322_0001, dagName=[B048CDD5A8E84A8995BDB2B2311D0BCC/DD493CA5713E4EF4BC10680AB04709A0] every/(1/1) ...semblyplatform/-1_ARGS_-1
2016-01-14 17:16:30,469 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [B048CDD5A8E84A8995BDB2B2311D0BCC/DD493CA5713E4EF4BC10680AB04709A0] every/(1/1) ...semblyplatform/-1_ARGS_-1
2016-01-14 17:16:30,470 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791790322_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/B048CDD5A8E84A8995BDB2B2311D0BCC/DD493CA5713E4EF4BC10680AB04709A0/.tez/application_1452791790322_0001/localmode-log-dir/dag_1452791790322_0001_1.dot
2016-01-14 17:16:30,470 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/B048CDD5A8E84A8995BDB2B2311D0BCC/DD493CA5713E4EF4BC10680AB04709A0/.tez/application_1452791790322_0001/localmode-log-dir/dag_1452791790322_0001_1-tez-dag.pb.txt
2016-01-14 17:16:30,485 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791790322_0001_1
2016-01-14 17:16:30,485 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:30,485 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [B048CDD5A8E84A8995BDB2B2311D0BCC/DD493CA5713E4EF4BC10680AB04709A0] every/(1/1) ...semblyplatform/-1_ARGS_-1
2016-01-14 17:16:30 Running Dag: dag_1452791790322_0001_1
2016-01-14 17:16:30,499 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791790322_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791790322_0001_1, submitTime=1452791790469
2016-01-14 17:16:30,499 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:30,501 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex D63E33BA5EB2415690D15E48EF80BE0B
2016-01-14 17:16:30,501 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:30,502 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791790322_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791790322_0001_1, initTime=1452791790499
2016-01-14 17:16:30,502 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:30,502 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791790322_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:30,503 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791790322_0001_1][Event:DAG_STARTED]: dagID=dag_1452791790322_0001_1, startTime=1452791790503
2016-01-14 17:16:30,503 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:30,503 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791790322_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:30,503 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: AB5C7DB027F545099A630757298290C7 : {BA9A5A6104D64CCAB825F00673770651={InputName=BA9A5A6104D64CCAB825F00673770651}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:30,504 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: BA9A5A6104D64CCAB825F00673770651, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:30,504 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791790322_0001_1_00 [AB5C7DB027F545099A630757298290C7]
2016-01-14 17:16:30,504 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:30,504 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791790322_0001_1_00 [AB5C7DB027F545099A630757298290C7]
2016-01-14 17:16:30,504 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791790322_0001_1_00 [AB5C7DB027F545099A630757298290C7]
2016-01-14 17:16:30,505 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791790322_0001_1_00 [AB5C7DB027F545099A630757298290C7]. Starting root input initializers: 1
2016-01-14 17:16:30,505 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...semblyplatform/-1_ARGS_-1, applicationId=application_1452791790322_0001, dagName=[B048CDD5A8E84A8995BDB2B2311D0BCC/DD493CA5713E4EF4BC10680AB04709A0] every/(1/1) ...semblyplatform/-1_ARGS_-1
2016-01-14 17:16:30,505 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791790322_0001_1_00 [AB5C7DB027F545099A630757298290C7] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:30,506 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791790322_0001_1_01 [D63E33BA5EB2415690D15E48EF80BE0B]
2016-01-14 17:16:30,507 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: BA9A5A6104D64CCAB825F00673770651 on vertex vertex_1452791790322_0001_1_00 [AB5C7DB027F545099A630757298290C7]
2016-01-14 17:16:30,509 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:30,509 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:30,509 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791790322_0001_1_01 [D63E33BA5EB2415690D15E48EF80BE0B]
2016-01-14 17:16:30,509 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791790322_0001_1_01 [D63E33BA5EB2415690D15E48EF80BE0B]
2016-01-14 17:16:30,510 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791790322_0001_1_01 [D63E33BA5EB2415690D15E48EF80BE0B]
2016-01-14 17:16:30,510 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=07C8F37EC77240019082C9FE1464854A, vertexId=vertex_1452791790322_0001_1_01 [D63E33BA5EB2415690D15E48EF80BE0B], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:30,509 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input BA9A5A6104D64CCAB825F00673770651 asking for 1 tasks. Headroom: 728 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:30,510 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:30,510 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=07C8F37EC77240019082C9FE1464854A, vertexId=vertex_1452791790322_0001_1_01 [D63E33BA5EB2415690D15E48EF80BE0B]
2016-01-14 17:16:30,511 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:30,512 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=07C8F37EC77240019082C9FE1464854A, vertexName=D63E33BA5EB2415690D15E48EF80BE0B, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:30,513 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=07C8F37EC77240019082C9FE1464854A, vertexId=vertex_1452791790322_0001_1_01 [D63E33BA5EB2415690D15E48EF80BE0B]
2016-01-14 17:16:30,514 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:30,514 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: BA9A5A6104D64CCAB825F00673770651 on vertex vertex_1452791790322_0001_1_00 [AB5C7DB027F545099A630757298290C7]
2016-01-14 17:16:30,514 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791790322_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D63E33BA5EB2415690D15E48EF80BE0B, vertexId=vertex_1452791790322_0001_1_01, initRequestedTime=1452791790506, initedTime=1452791790514, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:30,514 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:30,515 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791790322_0001_1_01 [D63E33BA5EB2415690D15E48EF80BE0B] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:30,515 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791790322_0001_1_00 [AB5C7DB027F545099A630757298290C7] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:30,515 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {BA9A5A6104D64CCAB825F00673770651=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:30,516 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791790322_0001_1_00 [AB5C7DB027F545099A630757298290C7] parallelism set to 1
2016-01-14 17:16:30,516 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791790322_0001_1_00 [AB5C7DB027F545099A630757298290C7] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:30,516 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791790322_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791790322_0001_1_00, eventCount=1
2016-01-14 17:16:30,517 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:30,517 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791790322_0001_1][Event:VERTEX_INITIALIZED]: vertexName=AB5C7DB027F545099A630757298290C7, vertexId=vertex_1452791790322_0001_1_00, initRequestedTime=1452791790503, initedTime=1452791790517, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:30,518 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791790322_0001_1_00 [AB5C7DB027F545099A630757298290C7] with distanceFromRoot: 0
2016-01-14 17:16:30,518 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791790322_0001_1_00 [AB5C7DB027F545099A630757298290C7] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:30,518 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in AB5C7DB027F545099A630757298290C7
2016-01-14 17:16:30,518 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791790322_0001_1_00 [AB5C7DB027F545099A630757298290C7]
2016-01-14 17:16:30,519 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:30,519 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791790322_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791790322_0001_1_00, startRequestedTime=1452791790518, startedTime=1452791790518
2016-01-14 17:16:30,520 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791790322_0001_1_00 [AB5C7DB027F545099A630757298290C7] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:30,520 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:30,520 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791790322_0001_1][Event:TASK_STARTED]: vertexName=AB5C7DB027F545099A630757298290C7, taskId=task_1452791790322_0001_1_00_000000, scheduledTime=1452791790520, launchTime=1452791790520
2016-01-14 17:16:30,521 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791790322_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:30,521 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791790322_0001_1_00 for vertex: vertex_1452791790322_0001_1_01 [D63E33BA5EB2415690D15E48EF80BE0B] numStartedSources: 1 numSources: 1
2016-01-14 17:16:30,521 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791790322_0001_1_01 [D63E33BA5EB2415690D15E48EF80BE0B] with distanceFromRoot: 1
2016-01-14 17:16:30,521 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791790322_0001_1_00 [AB5C7DB027F545099A630757298290C7]
2016-01-14 17:16:30,522 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: AB5C7DB027F545099A630757298290C7 in vertex: D63E33BA5EB2415690D15E48EF80BE0B
2016-01-14 17:16:30,522 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: D63E33BA5EB2415690D15E48EF80BE0B with 1 source tasks and 1 pending tasks
2016-01-14 17:16:30,522 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:30,523 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791790322_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791790322_0001_1_01, startRequestedTime=1452791790522, startedTime=1452791790522
2016-01-14 17:16:30,523 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791790322_0001_1_01 [D63E33BA5EB2415690D15E48EF80BE0B] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:30,523 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [B048CDD5A8E84A8995BDB2B2311D0BCC/DD493CA5713E4EF4BC10680AB04709A0] every/(1/1) ...semblyplatform/-1_ARGS_-1, VertexName: AB5C7DB027F545099A630757298290C7, VertexParallelism: 1, TaskAttemptID:attempt_1452791790322_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BA9A5A6104D64CCAB825F00673770651, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D63E33BA5EB2415690D15E48EF80BE0B, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:30,523 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791790322_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:30,524 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:30,524 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:30,525 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:30,525 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:30,525 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791790322_0001_00_000001
2016-01-14 17:16:30,526 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791790322_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:30,526 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791790322_0001, with dag id: dag_1452791790322_0001_1
2016-01-14 17:16:30,528 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791790322_0001_00_000001
2016-01-14 17:16:30,528 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:30,528 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791790322_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:30,528 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:30,529 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791790322_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791790322_0001_00_000001, launchTime=1452791790528
2016-01-14 17:16:30,530 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791790322_0001_00_000001
2016-01-14 17:16:30,530 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791790322_0001_1_00_000000_0] to container: [container_1452791790322_0001_00_000001]
2016-01-14 17:16:30,531 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791790322_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:30,531 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791790322_0001_1_00_000000_0] started. Is using containerId: [container_1452791790322_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:30,531 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791790322_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=AB5C7DB027F545099A630757298290C7, taskAttemptId=attempt_1452791790322_0001_1_00_000000_0, startTime=1452791790531, containerId=container_1452791790322_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791790322_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:30,531 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:30,532 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791790322_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:30,532 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791790322_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:30,532 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791790322_0001_00_000001 given task: attempt_1452791790322_0001_1_00_000000_0
2016-01-14 17:16:30,532 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791790322_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791790322_0001_1_00_000000_0
2016-01-14 17:16:30 Starting to run new task attempt: attempt_1452791790322_0001_1_00_000000_0
2016-01-14 17:16:30,533 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791790322_0001_1_00_000000_0
2016-01-14 17:16:30,533 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:30,533 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:30,533 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:30,533 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:30,533 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:30,533 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:30,534 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [B048CDD5A8E84A8995BDB2B2311D0BCC/DD493CA5713E4EF4BC10680AB04709A0] every/(1/1) ...semblyplatform/-1_ARGS_-1, VertexName: AB5C7DB027F545099A630757298290C7, VertexParallelism: 1, TaskAttemptID:attempt_1452791790322_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BA9A5A6104D64CCAB825F00673770651, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D63E33BA5EB2415690D15E48EF80BE0B, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:30,534 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=763363328
2016-01-14 17:16:30,534 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791790322_0001_1_00_000000_0
2016-01-14 17:16:30,535 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:30,535 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:30,535 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791790322_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:30,536 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=D63E33BA5EB2415690D15E48EF80BE0B, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:30,537 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:30,536 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=BA9A5A6104D64CCAB825F00673770651, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:30,536 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:30,537 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: D63E33BA5EB2415690D15E48EF80BE0B
2016-01-14 17:16:30,539 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:30,539 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: D63E33BA5EB2415690D15E48EF80BE0B
2016-01-14 17:16:30,539 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: D63E33BA5EB2415690D15E48EF80BE0B
2016-01-14 17:16:30,540 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:30,542 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:30,543 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: BA9A5A6104D64CCAB825F00673770651
2016-01-14 17:16:30,543 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: BA9A5A6104D64CCAB825F00673770651
2016-01-14 17:16:30,547 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:30,547 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: BA9A5A6104D64CCAB825F00673770651
2016-01-14 17:16:30,547 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: BA9A5A6104D64CCAB825F00673770651
2016-01-14 17:16:30,549 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: AB5C7DB027F545099A630757298290C7, ordinal: 0
2016-01-14 17:16:30,549 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AB5C7DB027F545099A630757298290C7, mem on start (mb), free: 313, total: 728, max: 728
2016-01-14 17:16:30,549 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:30,549 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:30,549 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:30,550 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:30,550 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:30,550 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:30,550 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:30,551 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:30,551 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:30,551 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 763363328, TotalAvailable: 511453429, TotalRequested/TotalJVMHeap:0.14
2016-01-14 17:16:30,551 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, D63E33BA5EB2415690D15E48EF80BE0B, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:30,552 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, BA9A5A6104D64CCAB825F00673770651, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:30,552 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:30,552 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: BA9A5A6104D64CCAB825F00673770651 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:30,552 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:30,553 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:30,552 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: BA9A5A6104D64CCAB825F00673770651
2016-01-14 17:16:30,553 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:30,553 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: BA9A5A6104D64CCAB825F00673770651
2016-01-14 17:16:30,554 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:30,554 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791790322_0001_1_00_000000_0
2016-01-14 17:16:30,554 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:30,555 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@f8495b0
2016-01-14 17:16:30,561 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:30,562 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: BA9A5A6104D64CCAB825F00673770651
2016-01-14 17:16:30,562 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 48643E58DB5646B9A163F530B4F26DB2
2016-01-14 17:16:30,562 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@f8495b0
2016-01-14 17:16:30,563 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:30,563 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:30,562 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 48643E58DB5646B9A163F530B4F26DB2
2016-01-14 17:16:30,565 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@5e30c94; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@7d4ecb90; comparator=cascading.tuple.hadoop.util.TupleComparator@40169b35; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:30,566 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:30,566 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:30,566 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:30,589 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:30,590 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:30,590 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:30,590 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:30,590 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:30,591 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:30,591 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:30,591 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: AB5C7DB027F545099A630757298290C7, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:30,594 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791790322_0001_1_00_000000_0
2016-01-14 17:16:30,594 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AB5C7DB027F545099A630757298290C7, mem on close (mb), free: 396, total: 728, max: 728
2016-01-14 17:16:30,594 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:30,594 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:30,594 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:30,595 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:30,597 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:30 Completed running task attempt: attempt_1452791790322_0001_1_00_000000_0
2016-01-14 17:16:30,601 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791790322_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:30,601 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791790322_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791790322_0001_1_00, eventCount=1
2016-01-14 17:16:30,602 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791790322_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=AB5C7DB027F545099A630757298290C7, taskAttemptId=attempt_1452791790322_0001_1_00_000000_0, startTime=1452791790531, finishTime=1452791790602, timeTaken=71, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=6, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=763363328, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791790591, Process_Duration=3, Process_End_Time=1452791790594, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:30,602 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791790322_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:30,602 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:30,602 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:30,602 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791790322_0001_1_00_000000_0
2016-01-14 17:16:30,603 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791790322_0001_1][Event:TASK_FINISHED]: vertexName=AB5C7DB027F545099A630757298290C7, taskId=task_1452791790322_0001_1_00_000000, startTime=1452791790531, finishTime=1452791790603, timeTaken=72, status=SUCCEEDED, successfulAttemptID=attempt_1452791790322_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=6, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=763363328, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791790591, Process_Duration=3, Process_End_Time=1452791790594, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:30,603 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:30,603 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791790322_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:30,605 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791790322_0001_1_00 [AB5C7DB027F545099A630757298290C7] : 1
2016-01-14 17:16:30,605 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791790322_0001_1_00 [AB5C7DB027F545099A630757298290C7]
2016-01-14 17:16:30,606 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:30,606 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791790322_0001_1][Event:VERTEX_FINISHED]: vertexName=AB5C7DB027F545099A630757298290C7, vertexId=vertex_1452791790322_0001_1_00, initRequestedTime=1452791790503, initedTime=1452791790517, startRequestedTime=1452791790518, startedTime=1452791790518, finishTime=1452791790605, timeTaken=87, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=6, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=763363328, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791790591, Process_Duration=3, Process_End_Time=1452791790594, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791790531, firstTasksToStart=[ task_1452791790322_0001_1_00_000000 ], lastTaskFinishTime=1452791790602, lastTasksToFinish=[ task_1452791790322_0001_1_00_000000 ], minTaskDuration=71, maxTaskDuration=71, avgTaskDuration=71.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791790322_0001_1_00_000000 ], longestDurationTasks=[ task_1452791790322_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:30,606 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791790322_0001_1_00 [AB5C7DB027F545099A630757298290C7] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:30,607 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791790322_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:30,607 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791790322_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791790322_0001_00_000001, stoppedTime=1452791790607, exitStatus=0
2016-01-14 17:16:30,607 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791790322_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:30,607 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791790322_0001_1_01 [D63E33BA5EB2415690D15E48EF80BE0B] attempt: attempt_1452791790322_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:30,607 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: D63E33BA5EB2415690D15E48EF80BE0B
2016-01-14 17:16:30,608 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791790322_0001_1_01 [D63E33BA5EB2415690D15E48EF80BE0B]
2016-01-14 17:16:30,608 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791790322_0001_1_00 [AB5C7DB027F545099A630757298290C7] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:30,608 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:30,608 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791790322_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:30,608 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791790322_0001_1][Event:TASK_STARTED]: vertexName=D63E33BA5EB2415690D15E48EF80BE0B, taskId=task_1452791790322_0001_1_01_000000, scheduledTime=1452791790608, launchTime=1452791790608
2016-01-14 17:16:30,608 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791790322_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:30,609 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791790322_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:30,609 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791790322_0001_1_01 [D63E33BA5EB2415690D15E48EF80BE0B]
2016-01-14 17:16:30,609 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [B048CDD5A8E84A8995BDB2B2311D0BCC/DD493CA5713E4EF4BC10680AB04709A0] every/(1/1) ...semblyplatform/-1_ARGS_-1, VertexName: D63E33BA5EB2415690D15E48EF80BE0B, VertexParallelism: 1, TaskAttemptID:attempt_1452791790322_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=AB5C7DB027F545099A630757298290C7, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=07C8F37EC77240019082C9FE1464854A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:30,609 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791790322_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:30,609 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:30,609 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=6, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=763363328, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791790591, Process_Duration=3, Process_End_Time=1452791790594, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:30,609 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:30,610 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:30,610 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:30,610 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791790322_0001_00_000002
2016-01-14 17:16:30,610 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791790322_0001_00_000001
2016-01-14 17:16:30,610 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791790322_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:30,610 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791790322_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:30,611 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791790322_0001_00_000001, Exiting
2016-01-14 17:16:30,611 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791790322_0001_00_000001
2016-01-14 17:16:30,611 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791790322_0001_00_000001
2016-01-14 17:16:30,610 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791790322_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:30,613 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791790322_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:30,613 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791790322_0001_00_000001 completed successfully
2016-01-14 17:16:30,613 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791790322_0001_00_000002
2016-01-14 17:16:30,614 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791790322_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:30,614 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 17:16:30,617 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791790322_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:30,617 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:30,618 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791790322_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791790322_0001_00_000002, launchTime=1452791790617
2016-01-14 17:16:30,619 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791790322_0001_00_000002
2016-01-14 17:16:30,619 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791790322_0001_1_01_000000_0] to container: [container_1452791790322_0001_00_000002]
2016-01-14 17:16:30,619 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791790322_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:30,619 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791790322_0001_00_000002 given task: attempt_1452791790322_0001_1_01_000000_0
2016-01-14 17:16:30,620 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791790322_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791790322_0001_1_01_000000_0
2016-01-14 17:16:30,620 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791790322_0001_1_01_000000_0] started. Is using containerId: [container_1452791790322_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:30,620 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791790322_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D63E33BA5EB2415690D15E48EF80BE0B, taskAttemptId=attempt_1452791790322_0001_1_01_000000_0, startTime=1452791790620, containerId=container_1452791790322_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791790322_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:30,620 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791790322_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:30,620 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:30,620 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791790322_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:30 Starting to run new task attempt: attempt_1452791790322_0001_1_01_000000_0
2016-01-14 17:16:30,622 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791790322_0001_1_01_000000_0
2016-01-14 17:16:30,622 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:30,622 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:30,622 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:30,622 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:30,623 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:30,623 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:30,623 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [B048CDD5A8E84A8995BDB2B2311D0BCC/DD493CA5713E4EF4BC10680AB04709A0] every/(1/1) ...semblyplatform/-1_ARGS_-1, VertexName: D63E33BA5EB2415690D15E48EF80BE0B, VertexParallelism: 1, TaskAttemptID:attempt_1452791790322_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=AB5C7DB027F545099A630757298290C7, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=07C8F37EC77240019082C9FE1464854A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:30,623 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=763363328
2016-01-14 17:16:30,624 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791790322_0001_1_01_000000_0
2016-01-14 17:16:30,625 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:30,625 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:30,625 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791790322_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:30,626 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=AB5C7DB027F545099A630757298290C7, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:30,626 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=07C8F37EC77240019082C9FE1464854A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:30,626 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:30,626 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:30,627 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:30,627 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 07C8F37EC77240019082C9FE1464854A
2016-01-14 17:16:30,628 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:30,628 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 07C8F37EC77240019082C9FE1464854A
2016-01-14 17:16:30,628 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: AB5C7DB027F545099A630757298290C7
2016-01-14 17:16:30,630 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 687027008, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:30,631 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:30,631 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 687027008, type: INPUT, componentVertexName: AB5C7DB027F545099A630757298290C7
2016-01-14 17:16:30,631 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:30,632 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: AB5C7DB027F545099A630757298290C7
2016-01-14 17:16:30,639 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D63E33BA5EB2415690D15E48EF80BE0B, ordinal: 1
2016-01-14 17:16:30,640 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D63E33BA5EB2415690D15E48EF80BE0B, mem on start (mb), free: 390, total: 728, max: 728
2016-01-14 17:16:30,640 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:30,640 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 07C8F37EC77240019082C9FE1464854A
2016-01-14 17:16:30,640 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:30,640 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:30,640 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:30,641 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:30,641 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:30,641 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:30,641 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:30,642 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:30,642 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:30,643 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 687027008, TotalRequestedScaled: 6.341787766153847E8, TotalJVMHeap: 763363328, TotalAvailable: 511453429, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:30,643 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 07C8F37EC77240019082C9FE1464854A, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:30,643 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, AB5C7DB027F545099A630757298290C7, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=687027008, allocated=511453429
2016-01-14 17:16:30,644 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:30,644 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: AB5C7DB027F545099A630757298290C7 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:30,644 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:30,644 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:30,644 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: AB5C7DB027F545099A630757298290C7
2016-01-14 17:16:30,645 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:30,646 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:30,646 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: AB5C7DB027F545099A630757298290C7 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:30,646 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=687027008, postMergeMem=0, RuntimeTotalAvailable=511453429. Updated to: ShuffleMem=511453429, postMergeMem: 0
2016-01-14 17:16:30,646 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=511453429, maxSingleShuffleLimit=127863360, mergeThreshold=460308096, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:30,647 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:30,648 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: AB5C7DB027F545099A630757298290C7
2016-01-14 17:16:30,648 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:30,648 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791790322_0001_1_01_000000_0
2016-01-14 17:16:30,649 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791790322_0001_1_00_000000_0_10092, runDuration: 61054, ]
2016-01-14 17:16:30,650 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791790322_0001_1_00_000000_0_10092] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791790650, TimeTaken=0, Rate=0.00 MB/s
2016-01-14 17:16:30,651 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : AB5C7DB027F545099A630757298290C7
2016-01-14 17:16:30,651 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:30,651 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:30,651 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [AB5C7DB027F545099A630757298290C7] #16 in 1ms
2016-01-14 17:16:30,652 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [AB5C7DB027F545099A630757298290C7] #16, status:false, isInterrupted:false
2016-01-14 17:16:30,652 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 48643E58DB5646B9A163F530B4F26DB2
2016-01-14 17:16:30,652 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_ARGS_-1"], id: 07C8F37EC77240019082C9FE1464854A
2016-01-14 17:16:30,652 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:30,653 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:30,653 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:30,653 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:30,653 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:30,654 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : AB5C7DB027F545099A630757298290C7
2016-01-14 17:16:30,654 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:30,654 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_ARGS_-1"]
2016-01-14 17:16:30,654 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 48643E58DB5646B9A163F530B4F26DB2, for 1 inputs
2016-01-14 17:16:30,655 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D63E33BA5EB2415690D15E48EF80BE0B, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:30,656 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@1455ce1e; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@d4f1e00
2016-01-14 17:16:30,660 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:30,660 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:30,660 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791790322_0001_1_01_000000_0
2016-01-14 17:16:30,661 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791790322_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:30,661 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791790322_0001_1_01_000000_0
2016-01-14 17:16:30,661 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D63E33BA5EB2415690D15E48EF80BE0B, mem on close (mb), free: 387, total: 728, max: 728
2016-01-14 17:16:30,662 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: AB5C7DB027F545099A630757298290C7
2016-01-14 17:16:30,662 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:30,662 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:30,663 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:30,663 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:30,663 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:30 Completed running task attempt: attempt_1452791790322_0001_1_01_000000_0
2016-01-14 17:16:30,664 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791790322_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:30,665 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:30,665 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791790322_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D63E33BA5EB2415690D15E48EF80BE0B, taskAttemptId=attempt_1452791790322_0001_1_01_000000_0, startTime=1452791790620, finishTime=1452791790665, timeTaken=45, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=763363328, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791790655, Process_Duration=6, Process_End_Time=1452791790661, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:30,665 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791790322_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:30,665 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=763363328, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791790655, Process_Duration=6, Process_End_Time=1452791790661, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 17:16:30,666 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791790322_0001_1_01_000000_0
2016-01-14 17:16:30,666 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:30,667 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791790322_0001_1][Event:TASK_FINISHED]: vertexName=D63E33BA5EB2415690D15E48EF80BE0B, taskId=task_1452791790322_0001_1_01_000000, startTime=1452791790620, finishTime=1452791790666, timeTaken=46, status=SUCCEEDED, successfulAttemptID=attempt_1452791790322_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=763363328, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791790655, Process_Duration=6, Process_End_Time=1452791790661, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:30,667 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791790322_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:30,666 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:30,667 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791790322_0001_1_01 [D63E33BA5EB2415690D15E48EF80BE0B] : 1
2016-01-14 17:16:30,667 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:30,667 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:30,668 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791790322_0001_00_000002
2016-01-14 17:16:30,667 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791790322_0001_1_01 [D63E33BA5EB2415690D15E48EF80BE0B]
2016-01-14 17:16:30,670 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:30,670 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791790322_0001_1][Event:VERTEX_FINISHED]: vertexName=D63E33BA5EB2415690D15E48EF80BE0B, vertexId=vertex_1452791790322_0001_1_01, initRequestedTime=1452791790506, initedTime=1452791790514, startRequestedTime=1452791790522, startedTime=1452791790522, finishTime=1452791790669, timeTaken=147, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=763363328, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791790655, Process_Duration=6, Process_End_Time=1452791790661, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452791790620, firstTasksToStart=[ task_1452791790322_0001_1_01_000000 ], lastTaskFinishTime=1452791790665, lastTasksToFinish=[ task_1452791790322_0001_1_01_000000 ], minTaskDuration=45, maxTaskDuration=45, avgTaskDuration=45.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791790322_0001_1_01_000000 ], longestDurationTasks=[ task_1452791790322_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:30,670 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791790322_0001_1_01 [D63E33BA5EB2415690D15E48EF80BE0B] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:30,671 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791790322_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:30,671 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:30,671 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791790322_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791790322_0001_00_000002, stoppedTime=1452791790671, exitStatus=0
2016-01-14 17:16:30,671 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791790322_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:30,671 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791790322_0001_1_01 [D63E33BA5EB2415690D15E48EF80BE0B] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:30,671 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791790322_0001_1
2016-01-14 17:16:30,672 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791790322_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452791790322_0001_1
2016-01-14 17:16:30,672 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 07C8F37EC77240019082C9FE1464854A for vertex: vertex_1452791790322_0001_1_01 [D63E33BA5EB2415690D15E48EF80BE0B]
2016-01-14 17:16:30,678 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791790322_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 17:16:30,678 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791790322_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:30,679 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791790322_0001_00_000002, Exiting
2016-01-14 17:16:30,679 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791790322_0001_00_000002
2016-01-14 17:16:30,679 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791790322_0001_00_000002
2016-01-14 17:16:30,679 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791790322_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:30,679 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791790322_0001_00_000002 completed successfully
2016-01-14 17:16:30,680 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452791790322_0001_1_00 [AB5C7DB027F545099A630757298290C7]
2016-01-14 17:16:30,681 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791790322_0001_1, queueSize=0
2016-01-14 17:16:30,682 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:30,682 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791790322_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791790322_0001_1, startTime=1452791790503, finishTime=1452791790680, timeTaken=177, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=465, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=6, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1526726656, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905583581246, Process_Duration=9, Process_End_Time=2905583581255, Read_Duration=1, Tuples_Read=40, Tuples_Written=40, Write_Duration=3, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 17:16:30,682 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791790322_0001_1 finished with state: SUCCEEDED
2016-01-14 17:16:30,682 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791790322_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:30,682 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452791790322_0001_00_000002
2016-01-14 17:16:30,682 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791790322_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:16:30 Completed Dag: dag_1452791790322_0001_1
2016-01-14 17:16:30,683 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791790322_0001_1, dagState=SUCCEEDED
2016-01-14 17:16:30,683 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791790322_0001_1_post
2016-01-14 17:16:30,683 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:30,683 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:30,690 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...semblyplatform/-1_ARGS_-1, applicationId=application_1452791790322_0001
2016-01-14 17:16:30,691 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:30,692 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:30,692 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:30,692 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:30,692 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:30,692 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:30,693 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:30,693 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:30,693 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:30,693 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:30,694 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:30,694 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:30,694 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:30,698 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/B048CDD5A8E84A8995BDB2B2311D0BCC/DD493CA5713E4EF4BC10680AB04709A0/.tez/application_1452791790322_0001
2016-01-14 17:16:30,699 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.400
2016-01-14 17:16:30,766 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_ARGS_-1
2016-01-14 17:16:30,787 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:30,788 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:30,817 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.005
2016-01-14 17:16:30,817 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.005
2016-01-14 17:16:30,818 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 17:16:30,819 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 17:16:30,819 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_ARGS_'field'
2016-01-14 17:16:30,840 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:30,841 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:30,874 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-01-14 17:16:30,880 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.016
2016-01-14 17:16:30,881 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:30,881 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:30,881 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:30,895 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:30,901 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_ARGS_RESULTS
2016-01-14 17:16:30,944 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:30,944 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:30,944 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_ARGS_RESULTS"]
2016-01-14 17:16:30,944 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:30,944 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:30,944 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:30,945 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...yplatform/-1_ARGS_RESULTS
2016-01-14 17:16:30,945 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:30,953 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 17:16:30,953 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 17:16:30,957 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:30,961 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:30,962 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:30,962 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:30,971 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/EB12EB07AC0C4B1E8DDD06224668E810/F7967417FCCE40558E8D9D695F6D6167/.tez/application_1452791790961_0001 doesn't exist and is created
2016-01-14 17:16:30,989 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:30,990 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:30,990 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/EB12EB07AC0C4B1E8DDD06224668E810/F7967417FCCE40558E8D9D695F6D6167/.tez/application_1452791790961_0001
2016-01-14 17:16:30,993 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791790961_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:31,002 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:31,003 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:31,004 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:31,004 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:31,004 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/EB12EB07AC0C4B1E8DDD06224668E810/F7967417FCCE40558E8D9D695F6D6167/.tez/application_1452791790961_0001/localmode-log-dir
2016-01-14 17:16:31,004 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/EB12EB07AC0C4B1E8DDD06224668E810/F7967417FCCE40558E8D9D695F6D6167/.tez/application_1452791790961_0001/localmode-log-dir/history.txt.appattempt_1452791790961_0001_000000, maxErrors=10
2016-01-14 17:16:31,005 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:31,005 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791790961_0001_000000, appSubmitTime=1452791790992, launchTime=1452791790992
2016-01-14 17:16:31,008 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:31,011 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:31,012 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:31,012 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:31,013 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:47309
2016-01-14 17:16:31,019 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:31,022 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791790961_0001_000000, startTime=1452791791022
2016-01-14 17:16:31,022 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:31,023 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:31,090 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:31,090 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:31,090 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...yplatform/-1_ARGS_RESULTS, applicationId=application_1452791790961_0001, dagName=[EB12EB07AC0C4B1E8DDD06224668E810/F7967417FCCE40558E8D9D695F6D6167] every/(1/1) ...yplatform/-1_ARGS_RESULTS
2016-01-14 17:16:31,105 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [EB12EB07AC0C4B1E8DDD06224668E810/F7967417FCCE40558E8D9D695F6D6167] every/(1/1) ...yplatform/-1_ARGS_RESULTS
2016-01-14 17:16:31,107 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791790961_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/EB12EB07AC0C4B1E8DDD06224668E810/F7967417FCCE40558E8D9D695F6D6167/.tez/application_1452791790961_0001/localmode-log-dir/dag_1452791790961_0001_1.dot
2016-01-14 17:16:31,108 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/EB12EB07AC0C4B1E8DDD06224668E810/F7967417FCCE40558E8D9D695F6D6167/.tez/application_1452791790961_0001/localmode-log-dir/dag_1452791790961_0001_1-tez-dag.pb.txt
2016-01-14 17:16:31,118 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791790961_0001_1
2016-01-14 17:16:31,118 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:31,118 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [EB12EB07AC0C4B1E8DDD06224668E810/F7967417FCCE40558E8D9D695F6D6167] every/(1/1) ...yplatform/-1_ARGS_RESULTS
2016-01-14 17:16:31 Running Dag: dag_1452791790961_0001_1
2016-01-14 17:16:31,135 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791790961_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791790961_0001_1, submitTime=1452791791105
2016-01-14 17:16:31,135 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:31,136 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex EE69754ED6C94EC19510CCCA43F00750
2016-01-14 17:16:31,136 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:31,137 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:31,137 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791790961_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791790961_0001_1, initTime=1452791791135
2016-01-14 17:16:31,138 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791790961_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:31,139 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791790961_0001_1][Event:DAG_STARTED]: dagID=dag_1452791790961_0001_1, startTime=1452791791138
2016-01-14 17:16:31,139 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:31,139 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791790961_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:31,139 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D24768B504D14281A7BF84F5943CDE1A : {D4E751BF34C6430495E191837453A521={InputName=D4E751BF34C6430495E191837453A521}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:31,139 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: D4E751BF34C6430495E191837453A521, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:31,139 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...yplatform/-1_ARGS_RESULTS, applicationId=application_1452791790961_0001, dagName=[EB12EB07AC0C4B1E8DDD06224668E810/F7967417FCCE40558E8D9D695F6D6167] every/(1/1) ...yplatform/-1_ARGS_RESULTS
2016-01-14 17:16:31,139 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:31,139 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791790961_0001_1_00 [D24768B504D14281A7BF84F5943CDE1A]
2016-01-14 17:16:31,140 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791790961_0001_1_00 [D24768B504D14281A7BF84F5943CDE1A]
2016-01-14 17:16:31,140 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791790961_0001_1_00 [D24768B504D14281A7BF84F5943CDE1A]
2016-01-14 17:16:31,140 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791790961_0001_1_00 [D24768B504D14281A7BF84F5943CDE1A]. Starting root input initializers: 1
2016-01-14 17:16:31,140 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791790961_0001_1_00 [D24768B504D14281A7BF84F5943CDE1A] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:31,141 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791790961_0001_1_01 [EE69754ED6C94EC19510CCCA43F00750]
2016-01-14 17:16:31,144 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:31,144 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791790961_0001_1_01 [EE69754ED6C94EC19510CCCA43F00750]
2016-01-14 17:16:31,144 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791790961_0001_1_01 [EE69754ED6C94EC19510CCCA43F00750]
2016-01-14 17:16:31,144 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791790961_0001_1_01 [EE69754ED6C94EC19510CCCA43F00750]
2016-01-14 17:16:31,145 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=3D6272AE57D44ACCAFA8D555C845FB95, vertexId=vertex_1452791790961_0001_1_01 [EE69754ED6C94EC19510CCCA43F00750], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:31,145 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=3D6272AE57D44ACCAFA8D555C845FB95, vertexId=vertex_1452791790961_0001_1_01 [EE69754ED6C94EC19510CCCA43F00750]
2016-01-14 17:16:31,147 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: D4E751BF34C6430495E191837453A521 on vertex vertex_1452791790961_0001_1_00 [D24768B504D14281A7BF84F5943CDE1A]
2016-01-14 17:16:31,147 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=3D6272AE57D44ACCAFA8D555C845FB95, vertexName=EE69754ED6C94EC19510CCCA43F00750, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:31,149 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=3D6272AE57D44ACCAFA8D555C845FB95, vertexId=vertex_1452791790961_0001_1_01 [EE69754ED6C94EC19510CCCA43F00750]
2016-01-14 17:16:31,150 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:31,150 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input D4E751BF34C6430495E191837453A521 asking for 1 tasks. Headroom: 728 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:31,153 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:31,153 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791790961_0001_1][Event:VERTEX_INITIALIZED]: vertexName=EE69754ED6C94EC19510CCCA43F00750, vertexId=vertex_1452791790961_0001_1_01, initRequestedTime=1452791791141, initedTime=1452791791152, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:31,153 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791790961_0001_1_01 [EE69754ED6C94EC19510CCCA43F00750] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:31,154 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:31,155 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:31,157 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:31,157 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: D4E751BF34C6430495E191837453A521 on vertex vertex_1452791790961_0001_1_00 [D24768B504D14281A7BF84F5943CDE1A]
2016-01-14 17:16:31,158 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791790961_0001_1_00 [D24768B504D14281A7BF84F5943CDE1A] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:31,158 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791790961_0001, with dag id: dag_1452791790961_0001_1
2016-01-14 17:16:31,159 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {D4E751BF34C6430495E191837453A521=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:31,160 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791790961_0001_1_00 [D24768B504D14281A7BF84F5943CDE1A] parallelism set to 1
2016-01-14 17:16:31,161 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791790961_0001_1_00 [D24768B504D14281A7BF84F5943CDE1A] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:31,161 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791790961_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791790961_0001_1_00, eventCount=1
2016-01-14 17:16:31,162 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:31,162 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791790961_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D24768B504D14281A7BF84F5943CDE1A, vertexId=vertex_1452791790961_0001_1_00, initRequestedTime=1452791791139, initedTime=1452791791162, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:31,163 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791790961_0001_1_00 [D24768B504D14281A7BF84F5943CDE1A] with distanceFromRoot: 0
2016-01-14 17:16:31,163 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791790961_0001_1_00 [D24768B504D14281A7BF84F5943CDE1A] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:31,163 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D24768B504D14281A7BF84F5943CDE1A
2016-01-14 17:16:31,163 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791790961_0001_1_00 [D24768B504D14281A7BF84F5943CDE1A]
2016-01-14 17:16:31,164 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:31,165 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791790961_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791790961_0001_1_00, startRequestedTime=1452791791163, startedTime=1452791791163
2016-01-14 17:16:31,165 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791790961_0001_1_00 [D24768B504D14281A7BF84F5943CDE1A] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:31,165 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:31,166 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791790961_0001_1][Event:TASK_STARTED]: vertexName=D24768B504D14281A7BF84F5943CDE1A, taskId=task_1452791790961_0001_1_00_000000, scheduledTime=1452791791165, launchTime=1452791791165
2016-01-14 17:16:31,166 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791790961_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:31,167 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791790961_0001_1_00 for vertex: vertex_1452791790961_0001_1_01 [EE69754ED6C94EC19510CCCA43F00750] numStartedSources: 1 numSources: 1
2016-01-14 17:16:31,167 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791790961_0001_1_01 [EE69754ED6C94EC19510CCCA43F00750] with distanceFromRoot: 1
2016-01-14 17:16:31,167 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791790961_0001_1_00 [D24768B504D14281A7BF84F5943CDE1A]
2016-01-14 17:16:31,169 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: D24768B504D14281A7BF84F5943CDE1A in vertex: EE69754ED6C94EC19510CCCA43F00750
2016-01-14 17:16:31,169 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: EE69754ED6C94EC19510CCCA43F00750 with 1 source tasks and 1 pending tasks
2016-01-14 17:16:31,169 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:31,170 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791790961_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791790961_0001_1_01, startRequestedTime=1452791791169, startedTime=1452791791169
2016-01-14 17:16:31,170 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791790961_0001_1_01 [EE69754ED6C94EC19510CCCA43F00750] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:31,170 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [EB12EB07AC0C4B1E8DDD06224668E810/F7967417FCCE40558E8D9D695F6D6167] every/(1/1) ...yplatform/-1_ARGS_RESULTS, VertexName: D24768B504D14281A7BF84F5943CDE1A, VertexParallelism: 1, TaskAttemptID:attempt_1452791790961_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D4E751BF34C6430495E191837453A521, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=EE69754ED6C94EC19510CCCA43F00750, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:31,170 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791790961_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:31,170 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:31,171 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:31,171 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:31,172 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:31,172 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791790961_0001_00_000001
2016-01-14 17:16:31,172 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791790961_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:31,174 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791790961_0001_00_000001
2016-01-14 17:16:31,174 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:31,175 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791790961_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:31,175 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:31,175 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791790961_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791790961_0001_00_000001, launchTime=1452791791175
2016-01-14 17:16:31,176 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791790961_0001_00_000001
2016-01-14 17:16:31,176 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791790961_0001_1_00_000000_0] to container: [container_1452791790961_0001_00_000001]
2016-01-14 17:16:31,176 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791790961_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:31,177 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791790961_0001_1_00_000000_0] started. Is using containerId: [container_1452791790961_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:31,177 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791790961_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D24768B504D14281A7BF84F5943CDE1A, taskAttemptId=attempt_1452791790961_0001_1_00_000000_0, startTime=1452791791177, containerId=container_1452791790961_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791790961_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:31,177 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791790961_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:31,177 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791790961_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:31,178 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:31,178 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791790961_0001_00_000001 given task: attempt_1452791790961_0001_1_00_000000_0
2016-01-14 17:16:31,178 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791790961_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791790961_0001_1_00_000000_0
2016-01-14 17:16:31 Starting to run new task attempt: attempt_1452791790961_0001_1_00_000000_0
2016-01-14 17:16:31,178 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791790961_0001_1_00_000000_0
2016-01-14 17:16:31,179 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:31,179 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:31,179 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:31,179 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:31,179 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:31,179 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:31,179 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [EB12EB07AC0C4B1E8DDD06224668E810/F7967417FCCE40558E8D9D695F6D6167] every/(1/1) ...yplatform/-1_ARGS_RESULTS, VertexName: D24768B504D14281A7BF84F5943CDE1A, VertexParallelism: 1, TaskAttemptID:attempt_1452791790961_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D4E751BF34C6430495E191837453A521, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=EE69754ED6C94EC19510CCCA43F00750, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:31,180 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=763363328
2016-01-14 17:16:31,180 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791790961_0001_1_00_000000_0
2016-01-14 17:16:31,180 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:31,180 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:31,182 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791790961_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:31,184 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=D4E751BF34C6430495E191837453A521, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:31,184 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:31,185 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: D4E751BF34C6430495E191837453A521
2016-01-14 17:16:31,185 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: D4E751BF34C6430495E191837453A521
2016-01-14 17:16:31,187 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:31,188 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: D4E751BF34C6430495E191837453A521
2016-01-14 17:16:31,188 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: D4E751BF34C6430495E191837453A521
2016-01-14 17:16:31,188 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:31,190 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=EE69754ED6C94EC19510CCCA43F00750, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:31,191 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:31,191 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:31,191 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: EE69754ED6C94EC19510CCCA43F00750
2016-01-14 17:16:31,194 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:31,194 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: EE69754ED6C94EC19510CCCA43F00750
2016-01-14 17:16:31,199 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: EE69754ED6C94EC19510CCCA43F00750
2016-01-14 17:16:31,203 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D24768B504D14281A7BF84F5943CDE1A, ordinal: 0
2016-01-14 17:16:31,203 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D24768B504D14281A7BF84F5943CDE1A, mem on start (mb), free: 278, total: 728, max: 728
2016-01-14 17:16:31,203 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:31,203 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:31,204 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:31,204 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:31,204 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:31,205 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:31,205 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:31,205 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:31,206 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:31,206 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 763363328, TotalAvailable: 511453429, TotalRequested/TotalJVMHeap:0.14
2016-01-14 17:16:31,206 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, D4E751BF34C6430495E191837453A521, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:31,206 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, EE69754ED6C94EC19510CCCA43F00750, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:31,207 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:31,207 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: D4E751BF34C6430495E191837453A521
2016-01-14 17:16:31,207 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:31,208 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: D4E751BF34C6430495E191837453A521
2016-01-14 17:16:31,208 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: D4E751BF34C6430495E191837453A521 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:31,208 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:31,209 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:31,209 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:31,209 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791790961_0001_1_00_000000_0
2016-01-14 17:16:31,213 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:31,213 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: D4E751BF34C6430495E191837453A521
2016-01-14 17:16:31,213 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: E3F515B8A3B14B10B302699C623D62A6
2016-01-14 17:16:31,213 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] E3F515B8A3B14B10B302699C623D62A6
2016-01-14 17:16:31,214 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@701e810f; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@62ab394a; comparator=cascading.tuple.hadoop.util.TupleComparator@573e5be1; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:31,214 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:31,214 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:31,214 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:31,237 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:31,238 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:31,238 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:31,239 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@54a24704
2016-01-14 17:16:31,239 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@54a24704
2016-01-14 17:16:31,239 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:31,239 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:31,239 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:31,242 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:31,242 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:31,243 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:31,244 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:31,244 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D24768B504D14281A7BF84F5943CDE1A, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:31,248 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791790961_0001_1_00_000000_0
2016-01-14 17:16:31,248 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D24768B504D14281A7BF84F5943CDE1A, mem on close (mb), free: 395, total: 729, max: 729
2016-01-14 17:16:31,249 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:31,249 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:31,249 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:31,250 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:31,255 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:31 Completed running task attempt: attempt_1452791790961_0001_1_00_000000_0
2016-01-14 17:16:31,258 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791790961_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:31,259 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791790961_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791790961_0001_1_00, eventCount=1
2016-01-14 17:16:31,260 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:31,260 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791790961_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D24768B504D14281A7BF84F5943CDE1A, taskAttemptId=attempt_1452791790961_0001_1_00_000000_0, startTime=1452791791177, finishTime=1452791791260, timeTaken=83, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=764411904, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791791245, Process_Duration=3, Process_End_Time=1452791791248, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:31,260 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791790961_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:31,260 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:31,260 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791790961_0001_1_00_000000_0
2016-01-14 17:16:31,261 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:31,261 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791790961_0001_1][Event:TASK_FINISHED]: vertexName=D24768B504D14281A7BF84F5943CDE1A, taskId=task_1452791790961_0001_1_00_000000, startTime=1452791791177, finishTime=1452791791261, timeTaken=84, status=SUCCEEDED, successfulAttemptID=attempt_1452791790961_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=764411904, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791791245, Process_Duration=3, Process_End_Time=1452791791248, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:31,261 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791790961_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:31,262 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791790961_0001_1_00 [D24768B504D14281A7BF84F5943CDE1A] : 1
2016-01-14 17:16:31,262 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791790961_0001_1_00 [D24768B504D14281A7BF84F5943CDE1A]
2016-01-14 17:16:31,263 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791790961_0001_1][Event:VERTEX_FINISHED]: vertexName=D24768B504D14281A7BF84F5943CDE1A, vertexId=vertex_1452791790961_0001_1_00, initRequestedTime=1452791791139, initedTime=1452791791162, startRequestedTime=1452791791163, startedTime=1452791791163, finishTime=1452791791262, timeTaken=99, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=764411904, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791791245, Process_Duration=3, Process_End_Time=1452791791248, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791791177, firstTasksToStart=[ task_1452791790961_0001_1_00_000000 ], lastTaskFinishTime=1452791791260, lastTasksToFinish=[ task_1452791790961_0001_1_00_000000 ], minTaskDuration=83, maxTaskDuration=83, avgTaskDuration=83.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791790961_0001_1_00_000000 ], longestDurationTasks=[ task_1452791790961_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:31,263 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791790961_0001_1_00 [D24768B504D14281A7BF84F5943CDE1A] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:31,263 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791790961_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:31,264 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791790961_0001_1_01 [EE69754ED6C94EC19510CCCA43F00750] attempt: attempt_1452791790961_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:31,264 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:31,264 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: EE69754ED6C94EC19510CCCA43F00750
2016-01-14 17:16:31,264 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791790961_0001_1_01 [EE69754ED6C94EC19510CCCA43F00750]
2016-01-14 17:16:31,265 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791790961_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791790961_0001_00_000001, stoppedTime=1452791791265, exitStatus=0
2016-01-14 17:16:31,266 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791790961_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:31,267 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791790961_0001_1_00 [D24768B504D14281A7BF84F5943CDE1A] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:31,265 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:31,267 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=764411904, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791791245, Process_Duration=3, Process_End_Time=1452791791248, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:31,269 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:31,269 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:31,269 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:31,269 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791790961_0001_00_000001
2016-01-14 17:16:31,271 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791790961_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:31,271 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791790961_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:31,268 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791790961_0001_1][Event:TASK_STARTED]: vertexName=EE69754ED6C94EC19510CCCA43F00750, taskId=task_1452791790961_0001_1_01_000000, scheduledTime=1452791791268, launchTime=1452791791268
2016-01-14 17:16:31,271 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791790961_0001_00_000001, Exiting
2016-01-14 17:16:31,272 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791790961_0001_00_000001
2016-01-14 17:16:31,272 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791790961_0001_00_000001
2016-01-14 17:16:31,273 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791790961_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:31,273 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791790961_0001_1_01 [EE69754ED6C94EC19510CCCA43F00750]
2016-01-14 17:16:31,273 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791790961_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:31,274 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791790961_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:31,274 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [EB12EB07AC0C4B1E8DDD06224668E810/F7967417FCCE40558E8D9D695F6D6167] every/(1/1) ...yplatform/-1_ARGS_RESULTS, VertexName: EE69754ED6C94EC19510CCCA43F00750, VertexParallelism: 1, TaskAttemptID:attempt_1452791790961_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D24768B504D14281A7BF84F5943CDE1A, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=3D6272AE57D44ACCAFA8D555C845FB95, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:31,274 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791790961_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:31,275 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:31,275 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791790961_0001_00_000002
2016-01-14 17:16:31,275 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791790961_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:31,277 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791790961_0001_00_000002
2016-01-14 17:16:31,278 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 17:16:31,280 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791790961_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:31,281 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791790961_0001_00_000001 completed successfully
2016-01-14 17:16:31,281 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791790961_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:31,283 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791790961_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:31,283 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:31,284 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791790961_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791790961_0001_00_000002, launchTime=1452791791283
2016-01-14 17:16:31,284 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791790961_0001_00_000002
2016-01-14 17:16:31,285 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791790961_0001_1_01_000000_0] to container: [container_1452791790961_0001_00_000002]
2016-01-14 17:16:31,286 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791790961_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:31,287 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791790961_0001_1_01_000000_0] started. Is using containerId: [container_1452791790961_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:31,287 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791790961_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=EE69754ED6C94EC19510CCCA43F00750, taskAttemptId=attempt_1452791790961_0001_1_01_000000_0, startTime=1452791791286, containerId=container_1452791790961_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791790961_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:31,287 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791790961_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:31,287 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791790961_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:31,287 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:31,288 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791790961_0001_00_000002 given task: attempt_1452791790961_0001_1_01_000000_0
2016-01-14 17:16:31,289 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791790961_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791790961_0001_1_01_000000_0
2016-01-14 17:16:31 Starting to run new task attempt: attempt_1452791790961_0001_1_01_000000_0
2016-01-14 17:16:31,290 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791790961_0001_1_01_000000_0
2016-01-14 17:16:31,290 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:31,290 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:31,290 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:31,290 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:31,290 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:31,290 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:31,290 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [EB12EB07AC0C4B1E8DDD06224668E810/F7967417FCCE40558E8D9D695F6D6167] every/(1/1) ...yplatform/-1_ARGS_RESULTS, VertexName: EE69754ED6C94EC19510CCCA43F00750, VertexParallelism: 1, TaskAttemptID:attempt_1452791790961_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D24768B504D14281A7BF84F5943CDE1A, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=3D6272AE57D44ACCAFA8D555C845FB95, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:31,291 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=764411904
2016-01-14 17:16:31,293 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791790961_0001_1_01_000000_0
2016-01-14 17:16:31,293 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:31,294 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:31,295 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791790961_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:31,296 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=D24768B504D14281A7BF84F5943CDE1A, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:31,296 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:31,297 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: D24768B504D14281A7BF84F5943CDE1A
2016-01-14 17:16:31,299 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 687970688, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:31,299 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:31,300 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 687970688, type: INPUT, componentVertexName: D24768B504D14281A7BF84F5943CDE1A
2016-01-14 17:16:31,300 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: D24768B504D14281A7BF84F5943CDE1A
2016-01-14 17:16:31,301 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:31,304 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:31,307 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=3D6272AE57D44ACCAFA8D555C845FB95, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:31,307 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:31,308 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 3D6272AE57D44ACCAFA8D555C845FB95
2016-01-14 17:16:31,308 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:31,309 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 3D6272AE57D44ACCAFA8D555C845FB95
2016-01-14 17:16:31,313 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: EE69754ED6C94EC19510CCCA43F00750, ordinal: 1
2016-01-14 17:16:31,314 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EE69754ED6C94EC19510CCCA43F00750, mem on start (mb), free: 389, total: 729, max: 729
2016-01-14 17:16:31,314 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:31,315 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:31,315 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:31,319 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:31,320 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 3D6272AE57D44ACCAFA8D555C845FB95
2016-01-14 17:16:31,320 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:31,320 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:31,320 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:31,320 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:31,320 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:31,321 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:31,321 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 687970688, TotalRequestedScaled: 6.350498658461539E8, TotalJVMHeap: 764411904, TotalAvailable: 512155975, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:31,321 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, D24768B504D14281A7BF84F5943CDE1A, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=687970688, allocated=512155975
2016-01-14 17:16:31,321 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 3D6272AE57D44ACCAFA8D555C845FB95, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:31,321 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:31,321 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: D24768B504D14281A7BF84F5943CDE1A being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:31,321 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:31,321 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:31,322 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: D24768B504D14281A7BF84F5943CDE1A
2016-01-14 17:16:31,322 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:31,323 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:31,323 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: D24768B504D14281A7BF84F5943CDE1A with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:31,323 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=687970688, postMergeMem=0, RuntimeTotalAvailable=512155975. Updated to: ShuffleMem=512155975, postMergeMem: 0
2016-01-14 17:16:31,323 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=512155975, maxSingleShuffleLimit=128038992, mergeThreshold=460940352, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:31,323 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:31,330 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: D24768B504D14281A7BF84F5943CDE1A
2016-01-14 17:16:31,330 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:31,333 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791790961_0001_1_01_000000_0
2016-01-14 17:16:31,336 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791790961_0001_1_00_000000_0_10099, runDuration: 65473, ]
2016-01-14 17:16:31,337 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:31,337 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: E3F515B8A3B14B10B302699C623D62A6
2016-01-14 17:16:31,338 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_ARGS_RESULTS"], id: 3D6272AE57D44ACCAFA8D555C845FB95
2016-01-14 17:16:31,338 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791790961_0001_1_00_000000_0_10099] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791791338, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 17:16:31,339 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : D24768B504D14281A7BF84F5943CDE1A
2016-01-14 17:16:31,338 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_ARGS_RESULTS"]
2016-01-14 17:16:31,339 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:31,340 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [D24768B504D14281A7BF84F5943CDE1A] #17 in 3ms
2016-01-14 17:16:31,340 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] E3F515B8A3B14B10B302699C623D62A6, for 1 inputs
2016-01-14 17:16:31,340 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [D24768B504D14281A7BF84F5943CDE1A] #17, status:false, isInterrupted:false
2016-01-14 17:16:31,341 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:31,341 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:31,341 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:31,341 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:31,342 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:31,342 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : D24768B504D14281A7BF84F5943CDE1A
2016-01-14 17:16:31,342 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:31,343 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: EE69754ED6C94EC19510CCCA43F00750, all 1 inputs ready in: 00:00:00.003
2016-01-14 17:16:31,345 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@5e1855f8; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@fb43d0b
2016-01-14 17:16:31,349 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:31,349 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:31,350 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791790961_0001_1_01_000000_0
2016-01-14 17:16:31,350 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791790961_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:31,351 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791790961_0001_1_01_000000_0
2016-01-14 17:16:31,352 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EE69754ED6C94EC19510CCCA43F00750, mem on close (mb), free: 386, total: 729, max: 729
2016-01-14 17:16:31,352 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: D24768B504D14281A7BF84F5943CDE1A
2016-01-14 17:16:31,353 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:31,353 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:31,354 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:31,354 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:31,354 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:31 Completed running task attempt: attempt_1452791790961_0001_1_01_000000_0
2016-01-14 17:16:31,355 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791790961_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:31,357 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791790961_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=EE69754ED6C94EC19510CCCA43F00750, taskAttemptId=attempt_1452791790961_0001_1_01_000000_0, startTime=1452791791286, finishTime=1452791791356, timeTaken=70, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=764411904, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=17, MERGE_PHASE_TIME=19, FIRST_EVENT_RECEIVED=13, LAST_EVENT_RECEIVED=13, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791791343, Process_Duration=8, Process_End_Time=1452791791351, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:31,357 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:31,357 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791790961_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:31,357 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791790961_0001_1_01_000000_0
2016-01-14 17:16:31,357 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:31,357 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:31,358 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791790961_0001_1][Event:TASK_FINISHED]: vertexName=EE69754ED6C94EC19510CCCA43F00750, taskId=task_1452791790961_0001_1_01_000000, startTime=1452791791286, finishTime=1452791791357, timeTaken=71, status=SUCCEEDED, successfulAttemptID=attempt_1452791790961_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=764411904, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=17, MERGE_PHASE_TIME=19, FIRST_EVENT_RECEIVED=13, LAST_EVENT_RECEIVED=13, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791791343, Process_Duration=8, Process_End_Time=1452791791351, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:31,358 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791790961_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:31,358 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791790961_0001_1_01 [EE69754ED6C94EC19510CCCA43F00750] : 1
2016-01-14 17:16:31,358 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791790961_0001_1_01 [EE69754ED6C94EC19510CCCA43F00750]
2016-01-14 17:16:31,359 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:31,359 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=764411904, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=17, MERGE_PHASE_TIME=19, FIRST_EVENT_RECEIVED=13, LAST_EVENT_RECEIVED=13][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791791343, Process_Duration=8, Process_End_Time=1452791791351, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 17:16:31,359 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:31,359 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:31,360 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791790961_0001_1][Event:VERTEX_FINISHED]: vertexName=EE69754ED6C94EC19510CCCA43F00750, vertexId=vertex_1452791790961_0001_1_01, initRequestedTime=1452791791141, initedTime=1452791791152, startRequestedTime=1452791791169, startedTime=1452791791169, finishTime=1452791791358, timeTaken=189, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=764411904, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=17, MERGE_PHASE_TIME=19, FIRST_EVENT_RECEIVED=13, LAST_EVENT_RECEIVED=13, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791791343, Process_Duration=8, Process_End_Time=1452791791351, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452791791286, firstTasksToStart=[ task_1452791790961_0001_1_01_000000 ], lastTaskFinishTime=1452791791356, lastTasksToFinish=[ task_1452791790961_0001_1_01_000000 ], minTaskDuration=70, maxTaskDuration=70, avgTaskDuration=70.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791790961_0001_1_01_000000 ], longestDurationTasks=[ task_1452791790961_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:31,360 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791790961_0001_1_01 [EE69754ED6C94EC19510CCCA43F00750] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:31,360 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791790961_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:31,360 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791790961_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791790961_0001_00_000002, stoppedTime=1452791791360, exitStatus=0
2016-01-14 17:16:31,360 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:31,360 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791790961_0001_00_000002
2016-01-14 17:16:31,360 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791790961_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:31,360 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791790961_0001_1_01 [EE69754ED6C94EC19510CCCA43F00750] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:31,361 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791790961_0001_1
2016-01-14 17:16:31,361 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791790961_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452791790961_0001_1
2016-01-14 17:16:31,361 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 3D6272AE57D44ACCAFA8D555C845FB95 for vertex: vertex_1452791790961_0001_1_01 [EE69754ED6C94EC19510CCCA43F00750]
2016-01-14 17:16:31,364 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791790961_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 17:16:31,364 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791790961_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:31,364 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791790961_0001_00_000002, Exiting
2016-01-14 17:16:31,364 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791790961_0001_00_000002
2016-01-14 17:16:31,365 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791790961_0001_00_000002
2016-01-14 17:16:31,365 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791790961_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:31,365 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791790961_0001_00_000002 completed successfully
2016-01-14 17:16:31,371 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452791790961_0001_1_00 [D24768B504D14281A7BF84F5943CDE1A]
2016-01-14 17:16:31,373 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791790961_0001_1, queueSize=0
2016-01-14 17:16:31,374 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:31,374 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791790961_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791790961_0001_1, startTime=1452791791138, finishTime=1452791791371, timeTaken=233, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=465, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1528823808, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=17, MERGE_PHASE_TIME=19, FIRST_EVENT_RECEIVED=13, LAST_EVENT_RECEIVED=13, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905583582588, Process_Duration=11, Process_End_Time=2905583582599, Read_Duration=2, Tuples_Read=40, Tuples_Written=40, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 17:16:31,375 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791790961_0001_1 finished with state: SUCCEEDED
2016-01-14 17:16:31,375 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791790961_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:31,379 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791790961_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:16:31,379 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452791790961_0001_00_000002
2016-01-14 17:16:31 Completed Dag: dag_1452791790961_0001_1
2016-01-14 17:16:31,380 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791790961_0001_1, dagState=SUCCEEDED
2016-01-14 17:16:31,380 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791790961_0001_1_post
2016-01-14 17:16:31,380 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:31,380 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:31,383 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...yplatform/-1_ARGS_RESULTS, applicationId=application_1452791790961_0001
2016-01-14 17:16:31,384 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:31,384 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:31,384 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:31,384 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:31,385 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:31,385 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:31,385 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:31,386 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:31,386 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:31,386 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:31,385 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:31,386 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:31,387 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:31,391 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/EB12EB07AC0C4B1E8DDD06224668E810/F7967417FCCE40558E8D9D695F6D6167/.tez/application_1452791790961_0001
2016-01-14 17:16:31,391 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.447
2016-01-14 17:16:31,472 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_ARGS_RESULTS
2016-01-14 17:16:31,493 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:31,493 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:31,521 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.005
2016-01-14 17:16:31,521 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.005
2016-01-14 17:16:31,521 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 17:16:31,522 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 17:16:31,522 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_ARGS_ALL
2016-01-14 17:16:31,543 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:31,544 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:31,581 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-01-14 17:16:31,588 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.019
2016-01-14 17:16:31,588 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:31,588 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:31,588 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:31,602 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 17:16:31,602 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 17:16:31,602 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:31,609 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_ARGS_REPLACE
2016-01-14 17:16:31,652 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:31,652 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:31,652 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_ARGS_REPLACE"]
2016-01-14 17:16:31,652 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:31,653 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:31,653 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:31,653 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...yplatform/-1_ARGS_REPLACE
2016-01-14 17:16:31,653 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:31,665 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:31,669 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:31,670 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:31,670 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:31,678 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/C94FDAC276EE458C91E08A71A441F6C1/ED8D27EEBD5748D4AD43C9F7BB6EC883/.tez/application_1452791791670_0001 doesn't exist and is created
2016-01-14 17:16:31,694 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:31,694 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:31,694 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/C94FDAC276EE458C91E08A71A441F6C1/ED8D27EEBD5748D4AD43C9F7BB6EC883/.tez/application_1452791791670_0001
2016-01-14 17:16:31,697 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791791670_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:31,706 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:31,707 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:31,708 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:31,708 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:31,708 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/C94FDAC276EE458C91E08A71A441F6C1/ED8D27EEBD5748D4AD43C9F7BB6EC883/.tez/application_1452791791670_0001/localmode-log-dir
2016-01-14 17:16:31,709 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/C94FDAC276EE458C91E08A71A441F6C1/ED8D27EEBD5748D4AD43C9F7BB6EC883/.tez/application_1452791791670_0001/localmode-log-dir/history.txt.appattempt_1452791791670_0001_000000, maxErrors=10
2016-01-14 17:16:31,709 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:31,710 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791791670_0001_000000, appSubmitTime=1452791791697, launchTime=1452791791697
2016-01-14 17:16:31,712 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:31,714 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:31,715 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:31,715 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:31,716 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:54665
2016-01-14 17:16:31,719 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:31,721 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791791670_0001_000000, startTime=1452791791721
2016-01-14 17:16:31,721 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:31,721 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:31,794 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:31,794 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:31,794 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...yplatform/-1_ARGS_REPLACE, applicationId=application_1452791791670_0001, dagName=[C94FDAC276EE458C91E08A71A441F6C1/ED8D27EEBD5748D4AD43C9F7BB6EC883] every/(1/1) ...yplatform/-1_ARGS_REPLACE
2016-01-14 17:16:31,809 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [C94FDAC276EE458C91E08A71A441F6C1/ED8D27EEBD5748D4AD43C9F7BB6EC883] every/(1/1) ...yplatform/-1_ARGS_REPLACE
2016-01-14 17:16:31,810 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791791670_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/C94FDAC276EE458C91E08A71A441F6C1/ED8D27EEBD5748D4AD43C9F7BB6EC883/.tez/application_1452791791670_0001/localmode-log-dir/dag_1452791791670_0001_1.dot
2016-01-14 17:16:31,811 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/C94FDAC276EE458C91E08A71A441F6C1/ED8D27EEBD5748D4AD43C9F7BB6EC883/.tez/application_1452791791670_0001/localmode-log-dir/dag_1452791791670_0001_1-tez-dag.pb.txt
2016-01-14 17:16:31,817 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791791670_0001_1
2016-01-14 17:16:31,817 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:31,817 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [C94FDAC276EE458C91E08A71A441F6C1/ED8D27EEBD5748D4AD43C9F7BB6EC883] every/(1/1) ...yplatform/-1_ARGS_REPLACE
2016-01-14 17:16:31 Running Dag: dag_1452791791670_0001_1
2016-01-14 17:16:31,830 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:31,831 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791791670_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791791670_0001_1, submitTime=1452791791809
2016-01-14 17:16:31,832 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex B9D3408C4ABB4A5293B9D48B233CE1AC
2016-01-14 17:16:31,832 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:31,832 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:31,833 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791791670_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791791670_0001_1, initTime=1452791791831
2016-01-14 17:16:31,833 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791791670_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:31,834 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791791670_0001_1][Event:DAG_STARTED]: dagID=dag_1452791791670_0001_1, startTime=1452791791833
2016-01-14 17:16:31,835 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:31,835 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791791670_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:31,835 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:31,835 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: E4C7C29576FD46FE8CA1A1823BCC4461 : {9EC9788B5A5547F08AE6EAE9442883E3={InputName=9EC9788B5A5547F08AE6EAE9442883E3}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:31,835 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 9EC9788B5A5547F08AE6EAE9442883E3, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:31,835 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...yplatform/-1_ARGS_REPLACE, applicationId=application_1452791791670_0001, dagName=[C94FDAC276EE458C91E08A71A441F6C1/ED8D27EEBD5748D4AD43C9F7BB6EC883] every/(1/1) ...yplatform/-1_ARGS_REPLACE
2016-01-14 17:16:31,835 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791791670_0001_1_00 [E4C7C29576FD46FE8CA1A1823BCC4461]
2016-01-14 17:16:31,837 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791791670_0001_1_00 [E4C7C29576FD46FE8CA1A1823BCC4461]
2016-01-14 17:16:31,838 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791791670_0001_1_00 [E4C7C29576FD46FE8CA1A1823BCC4461]
2016-01-14 17:16:31,838 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791791670_0001_1_00 [E4C7C29576FD46FE8CA1A1823BCC4461]. Starting root input initializers: 1
2016-01-14 17:16:31,838 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791791670_0001_1_00 [E4C7C29576FD46FE8CA1A1823BCC4461] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:31,838 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791791670_0001_1_01 [B9D3408C4ABB4A5293B9D48B233CE1AC]
2016-01-14 17:16:31,839 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 9EC9788B5A5547F08AE6EAE9442883E3 on vertex vertex_1452791791670_0001_1_00 [E4C7C29576FD46FE8CA1A1823BCC4461]
2016-01-14 17:16:31,841 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:31,841 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:31,841 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791791670_0001_1_01 [B9D3408C4ABB4A5293B9D48B233CE1AC]
2016-01-14 17:16:31,842 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 9EC9788B5A5547F08AE6EAE9442883E3 asking for 1 tasks. Headroom: 729 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:31,842 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791791670_0001_1_01 [B9D3408C4ABB4A5293B9D48B233CE1AC]
2016-01-14 17:16:31,842 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791791670_0001_1_01 [B9D3408C4ABB4A5293B9D48B233CE1AC]
2016-01-14 17:16:31,842 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=D1C92915FAA845C589000CA0D49B24C4, vertexId=vertex_1452791791670_0001_1_01 [B9D3408C4ABB4A5293B9D48B233CE1AC], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:31,842 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:31,843 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=D1C92915FAA845C589000CA0D49B24C4, vertexId=vertex_1452791791670_0001_1_01 [B9D3408C4ABB4A5293B9D48B233CE1AC]
2016-01-14 17:16:31,843 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:31,845 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:31,845 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=D1C92915FAA845C589000CA0D49B24C4, vertexName=B9D3408C4ABB4A5293B9D48B233CE1AC, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:31,845 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 9EC9788B5A5547F08AE6EAE9442883E3 on vertex vertex_1452791791670_0001_1_00 [E4C7C29576FD46FE8CA1A1823BCC4461]
2016-01-14 17:16:31,845 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=D1C92915FAA845C589000CA0D49B24C4, vertexId=vertex_1452791791670_0001_1_01 [B9D3408C4ABB4A5293B9D48B233CE1AC]
2016-01-14 17:16:31,846 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791791670_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B9D3408C4ABB4A5293B9D48B233CE1AC, vertexId=vertex_1452791791670_0001_1_01, initRequestedTime=1452791791838, initedTime=1452791791846, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:31,846 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:31,847 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791791670_0001_1_01 [B9D3408C4ABB4A5293B9D48B233CE1AC] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:31,847 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791791670_0001_1_00 [E4C7C29576FD46FE8CA1A1823BCC4461] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:31,847 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {9EC9788B5A5547F08AE6EAE9442883E3=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:31,847 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791791670_0001_1_00 [E4C7C29576FD46FE8CA1A1823BCC4461] parallelism set to 1
2016-01-14 17:16:31,847 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791791670_0001_1_00 [E4C7C29576FD46FE8CA1A1823BCC4461] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:31,848 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791791670_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791791670_0001_1_00, eventCount=1
2016-01-14 17:16:31,848 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:31,849 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791791670_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E4C7C29576FD46FE8CA1A1823BCC4461, vertexId=vertex_1452791791670_0001_1_00, initRequestedTime=1452791791835, initedTime=1452791791848, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:31,849 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791791670_0001_1_00 [E4C7C29576FD46FE8CA1A1823BCC4461] with distanceFromRoot: 0
2016-01-14 17:16:31,849 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791791670_0001_1_00 [E4C7C29576FD46FE8CA1A1823BCC4461] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:31,849 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in E4C7C29576FD46FE8CA1A1823BCC4461
2016-01-14 17:16:31,849 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791791670_0001_1_00 [E4C7C29576FD46FE8CA1A1823BCC4461]
2016-01-14 17:16:31,850 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:31,850 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791791670_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791791670_0001_1_00, startRequestedTime=1452791791849, startedTime=1452791791849
2016-01-14 17:16:31,850 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791791670_0001_1_00 [E4C7C29576FD46FE8CA1A1823BCC4461] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:31,851 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:31,851 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791791670_0001_1][Event:TASK_STARTED]: vertexName=E4C7C29576FD46FE8CA1A1823BCC4461, taskId=task_1452791791670_0001_1_00_000000, scheduledTime=1452791791851, launchTime=1452791791851
2016-01-14 17:16:31,851 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791791670_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:31,852 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791791670_0001_1_00 for vertex: vertex_1452791791670_0001_1_01 [B9D3408C4ABB4A5293B9D48B233CE1AC] numStartedSources: 1 numSources: 1
2016-01-14 17:16:31,852 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791791670_0001_1_01 [B9D3408C4ABB4A5293B9D48B233CE1AC] with distanceFromRoot: 1
2016-01-14 17:16:31,852 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791791670_0001_1_00 [E4C7C29576FD46FE8CA1A1823BCC4461]
2016-01-14 17:16:31,852 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: E4C7C29576FD46FE8CA1A1823BCC4461 in vertex: B9D3408C4ABB4A5293B9D48B233CE1AC
2016-01-14 17:16:31,853 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: B9D3408C4ABB4A5293B9D48B233CE1AC with 1 source tasks and 1 pending tasks
2016-01-14 17:16:31,853 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791791670_0001, with dag id: dag_1452791791670_0001_1
2016-01-14 17:16:31,853 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:31,854 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791791670_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791791670_0001_1_01, startRequestedTime=1452791791852, startedTime=1452791791852
2016-01-14 17:16:31,854 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791791670_0001_1_01 [B9D3408C4ABB4A5293B9D48B233CE1AC] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:31,855 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [C94FDAC276EE458C91E08A71A441F6C1/ED8D27EEBD5748D4AD43C9F7BB6EC883] every/(1/1) ...yplatform/-1_ARGS_REPLACE, VertexName: E4C7C29576FD46FE8CA1A1823BCC4461, VertexParallelism: 1, TaskAttemptID:attempt_1452791791670_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=9EC9788B5A5547F08AE6EAE9442883E3, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B9D3408C4ABB4A5293B9D48B233CE1AC, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:31,855 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791791670_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:31,855 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:31,856 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:31,856 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:31,856 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:31,857 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791791670_0001_00_000001
2016-01-14 17:16:31,857 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791791670_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:31,859 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791791670_0001_00_000001
2016-01-14 17:16:31,859 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:31,860 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791791670_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:31,860 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:31,861 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791791670_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791791670_0001_00_000001, launchTime=1452791791860
2016-01-14 17:16:31,861 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791791670_0001_00_000001
2016-01-14 17:16:31,862 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791791670_0001_1_00_000000_0] to container: [container_1452791791670_0001_00_000001]
2016-01-14 17:16:31,862 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791791670_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:31,862 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791791670_0001_1_00_000000_0] started. Is using containerId: [container_1452791791670_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:31,863 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791791670_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E4C7C29576FD46FE8CA1A1823BCC4461, taskAttemptId=attempt_1452791791670_0001_1_00_000000_0, startTime=1452791791862, containerId=container_1452791791670_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791791670_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:31,863 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791791670_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:31,863 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791791670_0001_00_000001 given task: attempt_1452791791670_0001_1_00_000000_0
2016-01-14 17:16:31,863 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:31,863 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791791670_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791791670_0001_1_00_000000_0
2016-01-14 17:16:31,863 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791791670_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:31 Starting to run new task attempt: attempt_1452791791670_0001_1_00_000000_0
2016-01-14 17:16:31,864 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791791670_0001_1_00_000000_0
2016-01-14 17:16:31,864 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:31,865 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:31,865 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:31,865 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:31,865 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:31,865 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:31,866 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [C94FDAC276EE458C91E08A71A441F6C1/ED8D27EEBD5748D4AD43C9F7BB6EC883] every/(1/1) ...yplatform/-1_ARGS_REPLACE, VertexName: E4C7C29576FD46FE8CA1A1823BCC4461, VertexParallelism: 1, TaskAttemptID:attempt_1452791791670_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=9EC9788B5A5547F08AE6EAE9442883E3, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B9D3408C4ABB4A5293B9D48B233CE1AC, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:31,866 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=764411904
2016-01-14 17:16:31,866 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791791670_0001_1_00_000000_0
2016-01-14 17:16:31,867 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:31,867 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:31,867 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791791670_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:31,868 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=9EC9788B5A5547F08AE6EAE9442883E3, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:31,868 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=B9D3408C4ABB4A5293B9D48B233CE1AC, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:31,868 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:31,868 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:31,868 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 9EC9788B5A5547F08AE6EAE9442883E3
2016-01-14 17:16:31,869 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 9EC9788B5A5547F08AE6EAE9442883E3
2016-01-14 17:16:31,869 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:31,869 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: B9D3408C4ABB4A5293B9D48B233CE1AC
2016-01-14 17:16:31,870 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:31,871 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 9EC9788B5A5547F08AE6EAE9442883E3
2016-01-14 17:16:31,871 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 9EC9788B5A5547F08AE6EAE9442883E3
2016-01-14 17:16:31,876 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:31,877 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:31,877 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: B9D3408C4ABB4A5293B9D48B233CE1AC
2016-01-14 17:16:31,878 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: B9D3408C4ABB4A5293B9D48B233CE1AC
2016-01-14 17:16:31,881 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E4C7C29576FD46FE8CA1A1823BCC4461, ordinal: 0
2016-01-14 17:16:31,881 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E4C7C29576FD46FE8CA1A1823BCC4461, mem on start (mb), free: 280, total: 729, max: 729
2016-01-14 17:16:31,881 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:31,881 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:31,881 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:31,881 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:31,882 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:31,882 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:31,882 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:31,882 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:31,882 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:31,882 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 764411904, TotalAvailable: 512155975, TotalRequested/TotalJVMHeap:0.14
2016-01-14 17:16:31,883 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 9EC9788B5A5547F08AE6EAE9442883E3, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:31,883 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, B9D3408C4ABB4A5293B9D48B233CE1AC, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:31,883 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:31,883 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 9EC9788B5A5547F08AE6EAE9442883E3 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:31,883 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:31,883 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:31,883 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 9EC9788B5A5547F08AE6EAE9442883E3
2016-01-14 17:16:31,884 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:31,884 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 9EC9788B5A5547F08AE6EAE9442883E3
2016-01-14 17:16:31,884 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:31,884 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791791670_0001_1_00_000000_0
2016-01-14 17:16:31,886 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:31,888 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@3d27e959
2016-01-14 17:16:31,888 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:31,889 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 9EC9788B5A5547F08AE6EAE9442883E3
2016-01-14 17:16:31,889 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 363407F4CB704143AA4F2DCA928E106C
2016-01-14 17:16:31,889 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@3d27e959
2016-01-14 17:16:31,890 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:31,890 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:31,889 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 363407F4CB704143AA4F2DCA928E106C
2016-01-14 17:16:31,890 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@64b9c7d6; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@60eee643; comparator=cascading.tuple.hadoop.util.TupleComparator@644f6a6c; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:31,890 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:31,891 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:31,891 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:31,916 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:31,916 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:31,916 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:31,916 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:31,917 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:31,917 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:31,917 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:31,918 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E4C7C29576FD46FE8CA1A1823BCC4461, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:31,920 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791791670_0001_1_00_000000_0
2016-01-14 17:16:31,920 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E4C7C29576FD46FE8CA1A1823BCC4461, mem on close (mb), free: 394, total: 729, max: 729
2016-01-14 17:16:31,921 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:31,921 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:31,921 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:31,921 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:31,923 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:31 Completed running task attempt: attempt_1452791791670_0001_1_00_000000_0
2016-01-14 17:16:31,926 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791791670_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:31,927 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=11, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=764411904, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791791918, Process_Duration=2, Process_End_Time=1452791791920, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:31,927 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791791670_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791791670_0001_1_00, eventCount=1
2016-01-14 17:16:31,927 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:31,927 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:31,928 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791791670_0001_00_000001
2016-01-14 17:16:31,928 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791791670_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E4C7C29576FD46FE8CA1A1823BCC4461, taskAttemptId=attempt_1452791791670_0001_1_00_000000_0, startTime=1452791791862, finishTime=1452791791927, timeTaken=65, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=11, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=764411904, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791791918, Process_Duration=2, Process_End_Time=1452791791920, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:31,928 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:31,928 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791791670_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:31,928 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:31,928 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791791670_0001_1_00_000000_0
2016-01-14 17:16:31,929 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:31,929 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791791670_0001_1][Event:TASK_FINISHED]: vertexName=E4C7C29576FD46FE8CA1A1823BCC4461, taskId=task_1452791791670_0001_1_00_000000, startTime=1452791791862, finishTime=1452791791928, timeTaken=66, status=SUCCEEDED, successfulAttemptID=attempt_1452791791670_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=11, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=764411904, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791791918, Process_Duration=2, Process_End_Time=1452791791920, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:31,929 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791791670_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:31,930 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791791670_0001_1_00 [E4C7C29576FD46FE8CA1A1823BCC4461] : 1
2016-01-14 17:16:31,930 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791791670_0001_1_00 [E4C7C29576FD46FE8CA1A1823BCC4461]
2016-01-14 17:16:31,930 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:31,931 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791791670_0001_1][Event:VERTEX_FINISHED]: vertexName=E4C7C29576FD46FE8CA1A1823BCC4461, vertexId=vertex_1452791791670_0001_1_00, initRequestedTime=1452791791835, initedTime=1452791791848, startRequestedTime=1452791791849, startedTime=1452791791849, finishTime=1452791791930, timeTaken=81, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=11, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=764411904, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791791918, Process_Duration=2, Process_End_Time=1452791791920, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791791862, firstTasksToStart=[ task_1452791791670_0001_1_00_000000 ], lastTaskFinishTime=1452791791927, lastTasksToFinish=[ task_1452791791670_0001_1_00_000000 ], minTaskDuration=65, maxTaskDuration=65, avgTaskDuration=65.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791791670_0001_1_00_000000 ], longestDurationTasks=[ task_1452791791670_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:31,931 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791791670_0001_1_00 [E4C7C29576FD46FE8CA1A1823BCC4461] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:31,931 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791791670_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:31,931 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791791670_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791791670_0001_00_000001, stoppedTime=1452791791931, exitStatus=0
2016-01-14 17:16:31,931 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:31,932 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791791670_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:31,932 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791791670_0001_1_01 [B9D3408C4ABB4A5293B9D48B233CE1AC] attempt: attempt_1452791791670_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:31,932 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: B9D3408C4ABB4A5293B9D48B233CE1AC
2016-01-14 17:16:31,932 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791791670_0001_1_01 [B9D3408C4ABB4A5293B9D48B233CE1AC]
2016-01-14 17:16:31,933 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791791670_0001_1_00 [E4C7C29576FD46FE8CA1A1823BCC4461] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:31,933 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791791670_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:31,933 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791791670_0001_1][Event:TASK_STARTED]: vertexName=B9D3408C4ABB4A5293B9D48B233CE1AC, taskId=task_1452791791670_0001_1_01_000000, scheduledTime=1452791791933, launchTime=1452791791933
2016-01-14 17:16:31,933 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:31,934 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791791670_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:31,934 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791791670_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:31,934 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791791670_0001_1_01 [B9D3408C4ABB4A5293B9D48B233CE1AC]
2016-01-14 17:16:31,935 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [C94FDAC276EE458C91E08A71A441F6C1/ED8D27EEBD5748D4AD43C9F7BB6EC883] every/(1/1) ...yplatform/-1_ARGS_REPLACE, VertexName: B9D3408C4ABB4A5293B9D48B233CE1AC, VertexParallelism: 1, TaskAttemptID:attempt_1452791791670_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E4C7C29576FD46FE8CA1A1823BCC4461, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=D1C92915FAA845C589000CA0D49B24C4, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:31,935 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791791670_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:31,935 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:31,936 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791791670_0001_00_000002
2016-01-14 17:16:31,936 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791791670_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:31,938 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791791670_0001_00_000002
2016-01-14 17:16:31,938 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:31,938 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791791670_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:31,938 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791791670_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:31,939 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791791670_0001_00_000001, Exiting
2016-01-14 17:16:31,939 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791791670_0001_00_000001
2016-01-14 17:16:31,939 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791791670_0001_00_000001
2016-01-14 17:16:31,939 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791791670_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:31,939 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:31,939 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791791670_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791791670_0001_00_000002, launchTime=1452791791939
2016-01-14 17:16:31,940 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791791670_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:31,940 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791791670_0001_00_000001 completed successfully
2016-01-14 17:16:31,940 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791791670_0001_00_000002
2016-01-14 17:16:31,941 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791791670_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:31,941 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791791670_0001_1_01_000000_0] to container: [container_1452791791670_0001_00_000002]
2016-01-14 17:16:31,941 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791791670_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:31,941 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791791670_0001_00_000002 given task: attempt_1452791791670_0001_1_01_000000_0
2016-01-14 17:16:31,942 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791791670_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791791670_0001_1_01_000000_0
2016-01-14 17:16:31,942 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791791670_0001_1_01_000000_0] started. Is using containerId: [container_1452791791670_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:31 Starting to run new task attempt: attempt_1452791791670_0001_1_01_000000_0
2016-01-14 17:16:31,942 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791791670_0001_1_01_000000_0
2016-01-14 17:16:31,942 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:31,942 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:31,942 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:31,942 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791791670_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B9D3408C4ABB4A5293B9D48B233CE1AC, taskAttemptId=attempt_1452791791670_0001_1_01_000000_0, startTime=1452791791942, containerId=container_1452791791670_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791791670_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:31,942 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:31,943 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:31,943 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:31,943 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [C94FDAC276EE458C91E08A71A441F6C1/ED8D27EEBD5748D4AD43C9F7BB6EC883] every/(1/1) ...yplatform/-1_ARGS_REPLACE, VertexName: B9D3408C4ABB4A5293B9D48B233CE1AC, VertexParallelism: 1, TaskAttemptID:attempt_1452791791670_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E4C7C29576FD46FE8CA1A1823BCC4461, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=D1C92915FAA845C589000CA0D49B24C4, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:31,943 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=764411904
2016-01-14 17:16:31,943 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791791670_0001_1_01_000000_0
2016-01-14 17:16:31,944 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:31,944 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:31,942 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:31,943 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791791670_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:31,944 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791791670_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:31,945 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=E4C7C29576FD46FE8CA1A1823BCC4461, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:31,945 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=D1C92915FAA845C589000CA0D49B24C4, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:31,945 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:31,946 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: E4C7C29576FD46FE8CA1A1823BCC4461
2016-01-14 17:16:31,945 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791791670_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:31,945 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:31,946 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:31,947 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: D1C92915FAA845C589000CA0D49B24C4
2016-01-14 17:16:31,947 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:31,948 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: D1C92915FAA845C589000CA0D49B24C4
2016-01-14 17:16:31,948 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 687970688, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:31,948 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:31,949 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 687970688, type: INPUT, componentVertexName: E4C7C29576FD46FE8CA1A1823BCC4461
2016-01-14 17:16:31,949 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: E4C7C29576FD46FE8CA1A1823BCC4461
2016-01-14 17:16:31,950 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:31,957 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B9D3408C4ABB4A5293B9D48B233CE1AC, ordinal: 1
2016-01-14 17:16:31,957 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B9D3408C4ABB4A5293B9D48B233CE1AC, mem on start (mb), free: 388, total: 729, max: 729
2016-01-14 17:16:31,957 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:31,957 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:31,957 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:31,959 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:31,959 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: D1C92915FAA845C589000CA0D49B24C4
2016-01-14 17:16:31,959 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:31,959 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:31,960 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:31,960 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:31,960 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:31,960 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:31,960 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 687970688, TotalRequestedScaled: 6.350498658461539E8, TotalJVMHeap: 764411904, TotalAvailable: 512155975, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:31,960 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, D1C92915FAA845C589000CA0D49B24C4, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:31,960 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, E4C7C29576FD46FE8CA1A1823BCC4461, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=687970688, allocated=512155975
2016-01-14 17:16:31,960 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:31,960 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: E4C7C29576FD46FE8CA1A1823BCC4461 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:31,960 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: E4C7C29576FD46FE8CA1A1823BCC4461
2016-01-14 17:16:31,960 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:31,961 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:31,961 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:31,962 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: E4C7C29576FD46FE8CA1A1823BCC4461 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:31,961 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:31,962 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=687970688, postMergeMem=0, RuntimeTotalAvailable=512155975. Updated to: ShuffleMem=512155975, postMergeMem: 0
2016-01-14 17:16:31,962 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=512155975, maxSingleShuffleLimit=128038992, mergeThreshold=460940352, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:31,963 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:31,963 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: E4C7C29576FD46FE8CA1A1823BCC4461
2016-01-14 17:16:31,963 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:31,964 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791791670_0001_1_01_000000_0
2016-01-14 17:16:31,964 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791791670_0001_1_00_000000_0_10105, runDuration: 55776, ]
2016-01-14 17:16:31,966 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791791670_0001_1_00_000000_0_10105] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791791966, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 17:16:31,966 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : E4C7C29576FD46FE8CA1A1823BCC4461
2016-01-14 17:16:31,966 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:31,966 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [E4C7C29576FD46FE8CA1A1823BCC4461] #18 in 1ms
2016-01-14 17:16:31,967 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [E4C7C29576FD46FE8CA1A1823BCC4461] #18, status:false, isInterrupted:false
2016-01-14 17:16:31,967 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:31,967 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:31,967 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:31,968 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:31,968 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:31,968 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : E4C7C29576FD46FE8CA1A1823BCC4461
2016-01-14 17:16:31,968 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:31,968 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:31,969 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 363407F4CB704143AA4F2DCA928E106C
2016-01-14 17:16:31,969 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_ARGS_REPLACE"], id: D1C92915FAA845C589000CA0D49B24C4
2016-01-14 17:16:31,969 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_ARGS_REPLACE"]
2016-01-14 17:16:31,970 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 363407F4CB704143AA4F2DCA928E106C, for 1 inputs
2016-01-14 17:16:31,970 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B9D3408C4ABB4A5293B9D48B233CE1AC, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:31,970 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@2158326; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@525534db
2016-01-14 17:16:31,976 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:31,976 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:31,977 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791791670_0001_1_01_000000_0
2016-01-14 17:16:31,977 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791791670_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:31,977 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791791670_0001_1_01_000000_0
2016-01-14 17:16:31,978 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B9D3408C4ABB4A5293B9D48B233CE1AC, mem on close (mb), free: 386, total: 729, max: 729
2016-01-14 17:16:31,978 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: E4C7C29576FD46FE8CA1A1823BCC4461
2016-01-14 17:16:31,978 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:31,978 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:31,979 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:31,979 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:31,979 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:31 Completed running task attempt: attempt_1452791791670_0001_1_01_000000_0
2016-01-14 17:16:31,980 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791791670_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:31,981 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=707, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=764411904, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791791970, Process_Duration=7, Process_End_Time=1452791791977, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 17:16:31,981 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791791670_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B9D3408C4ABB4A5293B9D48B233CE1AC, taskAttemptId=attempt_1452791791670_0001_1_01_000000_0, startTime=1452791791942, finishTime=1452791791981, timeTaken=39, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=707, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=764411904, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791791970, Process_Duration=7, Process_End_Time=1452791791977, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:31,981 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791791670_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:31,982 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:31,981 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:31,982 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791791670_0001_1_01_000000_0
2016-01-14 17:16:31,982 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791791670_0001_1][Event:TASK_FINISHED]: vertexName=B9D3408C4ABB4A5293B9D48B233CE1AC, taskId=task_1452791791670_0001_1_01_000000, startTime=1452791791942, finishTime=1452791791982, timeTaken=40, status=SUCCEEDED, successfulAttemptID=attempt_1452791791670_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=707, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=764411904, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791791970, Process_Duration=7, Process_End_Time=1452791791977, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:31,981 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:31,983 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791791670_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:31,982 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:31,983 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791791670_0001_1_01 [B9D3408C4ABB4A5293B9D48B233CE1AC] : 1
2016-01-14 17:16:31,983 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791791670_0001_1_01 [B9D3408C4ABB4A5293B9D48B233CE1AC]
2016-01-14 17:16:31,983 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:31,984 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:31,984 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791791670_0001_1][Event:VERTEX_FINISHED]: vertexName=B9D3408C4ABB4A5293B9D48B233CE1AC, vertexId=vertex_1452791791670_0001_1_01, initRequestedTime=1452791791838, initedTime=1452791791846, startRequestedTime=1452791791852, startedTime=1452791791852, finishTime=1452791791983, timeTaken=131, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=707, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=764411904, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791791970, Process_Duration=7, Process_End_Time=1452791791977, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452791791942, firstTasksToStart=[ task_1452791791670_0001_1_01_000000 ], lastTaskFinishTime=1452791791981, lastTasksToFinish=[ task_1452791791670_0001_1_01_000000 ], minTaskDuration=39, maxTaskDuration=39, avgTaskDuration=39.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791791670_0001_1_01_000000 ], longestDurationTasks=[ task_1452791791670_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:31,984 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791791670_0001_1_01 [B9D3408C4ABB4A5293B9D48B233CE1AC] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:31,984 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791791670_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:31,984 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791791670_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791791670_0001_00_000002, stoppedTime=1452791791984, exitStatus=0
2016-01-14 17:16:31,984 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791791670_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:31,984 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:31,984 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791791670_0001_1_01 [B9D3408C4ABB4A5293B9D48B233CE1AC] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:31,984 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791791670_0001_1
2016-01-14 17:16:31,985 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791791670_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452791791670_0001_1
2016-01-14 17:16:31,985 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: D1C92915FAA845C589000CA0D49B24C4 for vertex: vertex_1452791791670_0001_1_01 [B9D3408C4ABB4A5293B9D48B233CE1AC]
2016-01-14 17:16:31,984 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791791670_0001_00_000002
2016-01-14 17:16:31,987 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791791670_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 17:16:31,987 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791791670_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:31,987 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791791670_0001_00_000002, Exiting
2016-01-14 17:16:31,988 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791791670_0001_00_000002
2016-01-14 17:16:31,988 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791791670_0001_00_000002
2016-01-14 17:16:31,988 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791791670_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:31,988 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791791670_0001_00_000002 completed successfully
2016-01-14 17:16:31,992 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452791791670_0001_1_00 [E4C7C29576FD46FE8CA1A1823BCC4461]
2016-01-14 17:16:31,994 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791791670_0001_1, queueSize=0
2016-01-14 17:16:31,994 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:31,995 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791791670_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791791670_0001_1, startTime=1452791791833, finishTime=1452791791992, timeTaken=159, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=1040, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=11, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1528823808, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905583583888, Process_Duration=9, Process_End_Time=2905583583897, Read_Duration=4, Tuples_Read=40, Tuples_Written=40, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 17:16:31,995 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791791670_0001_1 finished with state: SUCCEEDED
2016-01-14 17:16:31,995 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791791670_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:31,996 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452791791670_0001_00_000002
2016-01-14 17:16:31,997 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791791670_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:16:31 Completed Dag: dag_1452791791670_0001_1
2016-01-14 17:16:31,999 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791791670_0001_1, dagState=SUCCEEDED
2016-01-14 17:16:31,999 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791791670_0001_1_post
2016-01-14 17:16:32,000 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:32,000 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:32,003 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...yplatform/-1_ARGS_REPLACE, applicationId=application_1452791791670_0001
2016-01-14 17:16:32,005 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:32,005 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:32,005 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:32,006 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:32,006 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:32,007 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:32,008 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:32,008 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:32,008 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:32,008 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:32,008 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:32,009 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:32,009 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:32,014 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/C94FDAC276EE458C91E08A71A441F6C1/ED8D27EEBD5748D4AD43C9F7BB6EC883/.tez/application_1452791791670_0001
2016-01-14 17:16:32,014 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.362
2016-01-14 17:16:32,092 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_ARGS_REPLACE
2016-01-14 17:16:32,113 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:32,114 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:32,146 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.008
2016-01-14 17:16:32,154 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-01-14 17:16:32,154 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:32,154 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:32,155 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:32,170 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:32,176 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_ARGS_SWAP
2016-01-14 17:16:32,221 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:32,221 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:32,221 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_ARGS_SWAP"]
2016-01-14 17:16:32,221 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:32,222 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:32,222 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:32,222 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...mblyplatform/-1_ARGS_SWAP
2016-01-14 17:16:32,222 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:32,233 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:32,238 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:32,238 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:32,238 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:32,248 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/863904F52F1C4D1DAF49D4E2E6A218CB/F617B39D5A2B426C88C1A640E91E9E31/.tez/application_1452791792238_0001 doesn't exist and is created
2016-01-14 17:16:32,263 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:32,263 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:32,264 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/863904F52F1C4D1DAF49D4E2E6A218CB/F617B39D5A2B426C88C1A640E91E9E31/.tez/application_1452791792238_0001
2016-01-14 17:16:32,267 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791792238_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:32,276 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:32,277 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:32,280 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:32,280 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:32,280 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/863904F52F1C4D1DAF49D4E2E6A218CB/F617B39D5A2B426C88C1A640E91E9E31/.tez/application_1452791792238_0001/localmode-log-dir
2016-01-14 17:16:32,281 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/863904F52F1C4D1DAF49D4E2E6A218CB/F617B39D5A2B426C88C1A640E91E9E31/.tez/application_1452791792238_0001/localmode-log-dir/history.txt.appattempt_1452791792238_0001_000000, maxErrors=10
2016-01-14 17:16:32,282 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:32,282 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791792238_0001_000000, appSubmitTime=1452791792267, launchTime=1452791792267
2016-01-14 17:16:32,285 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:32,288 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:32,288 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:32,290 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:32,292 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:47283
2016-01-14 17:16:32,294 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:32,296 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791792238_0001_000000, startTime=1452791792296
2016-01-14 17:16:32,296 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:32,297 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:32,364 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:32,364 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:32,364 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...mblyplatform/-1_ARGS_SWAP, applicationId=application_1452791792238_0001, dagName=[863904F52F1C4D1DAF49D4E2E6A218CB/F617B39D5A2B426C88C1A640E91E9E31] every/(1/1) ...mblyplatform/-1_ARGS_SWAP
2016-01-14 17:16:32,378 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [863904F52F1C4D1DAF49D4E2E6A218CB/F617B39D5A2B426C88C1A640E91E9E31] every/(1/1) ...mblyplatform/-1_ARGS_SWAP
2016-01-14 17:16:32,380 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791792238_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/863904F52F1C4D1DAF49D4E2E6A218CB/F617B39D5A2B426C88C1A640E91E9E31/.tez/application_1452791792238_0001/localmode-log-dir/dag_1452791792238_0001_1.dot
2016-01-14 17:16:32,380 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/863904F52F1C4D1DAF49D4E2E6A218CB/F617B39D5A2B426C88C1A640E91E9E31/.tez/application_1452791792238_0001/localmode-log-dir/dag_1452791792238_0001_1-tez-dag.pb.txt
2016-01-14 17:16:32,387 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791792238_0001_1
2016-01-14 17:16:32,387 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:32,388 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [863904F52F1C4D1DAF49D4E2E6A218CB/F617B39D5A2B426C88C1A640E91E9E31] every/(1/1) ...mblyplatform/-1_ARGS_SWAP
2016-01-14 17:16:32 Running Dag: dag_1452791792238_0001_1
2016-01-14 17:16:32,400 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:32,400 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791792238_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791792238_0001_1, submitTime=1452791792379
2016-01-14 17:16:32,401 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex E4FC9D14B84F442E980726892E2118DB
2016-01-14 17:16:32,401 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:32,403 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:32,403 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791792238_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791792238_0001_1, initTime=1452791792400
2016-01-14 17:16:32,403 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791792238_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:32,403 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791792238_0001_1][Event:DAG_STARTED]: dagID=dag_1452791792238_0001_1, startTime=1452791792403
2016-01-14 17:16:32,403 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:32,404 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791792238_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:32,404 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: E945B299A598457E82C6D04B580E094B : {3AFBE761E5384F56913198534B38AD75={InputName=3AFBE761E5384F56913198534B38AD75}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:32,404 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 3AFBE761E5384F56913198534B38AD75, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:32,404 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791792238_0001_1_00 [E945B299A598457E82C6D04B580E094B]
2016-01-14 17:16:32,404 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:32,404 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791792238_0001_1_00 [E945B299A598457E82C6D04B580E094B]
2016-01-14 17:16:32,404 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791792238_0001_1_00 [E945B299A598457E82C6D04B580E094B]
2016-01-14 17:16:32,404 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791792238_0001_1_00 [E945B299A598457E82C6D04B580E094B]. Starting root input initializers: 1
2016-01-14 17:16:32,405 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791792238_0001_1_00 [E945B299A598457E82C6D04B580E094B] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:32,405 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 3AFBE761E5384F56913198534B38AD75 on vertex vertex_1452791792238_0001_1_00 [E945B299A598457E82C6D04B580E094B]
2016-01-14 17:16:32,405 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791792238_0001_1_01 [E4FC9D14B84F442E980726892E2118DB]
2016-01-14 17:16:32,407 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...mblyplatform/-1_ARGS_SWAP, applicationId=application_1452791792238_0001, dagName=[863904F52F1C4D1DAF49D4E2E6A218CB/F617B39D5A2B426C88C1A640E91E9E31] every/(1/1) ...mblyplatform/-1_ARGS_SWAP
2016-01-14 17:16:32,409 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:32,409 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 3AFBE761E5384F56913198534B38AD75 asking for 1 tasks. Headroom: 729 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:32,410 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:32,410 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791792238_0001_1_01 [E4FC9D14B84F442E980726892E2118DB]
2016-01-14 17:16:32,410 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791792238_0001_1_01 [E4FC9D14B84F442E980726892E2118DB]
2016-01-14 17:16:32,410 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791792238_0001_1_01 [E4FC9D14B84F442E980726892E2118DB]
2016-01-14 17:16:32,411 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=C2A2B9E011AD4253AC38CAFE0A94F704, vertexId=vertex_1452791792238_0001_1_01 [E4FC9D14B84F442E980726892E2118DB], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:32,411 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=C2A2B9E011AD4253AC38CAFE0A94F704, vertexId=vertex_1452791792238_0001_1_01 [E4FC9D14B84F442E980726892E2118DB]
2016-01-14 17:16:32,410 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:32,412 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:32,413 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=C2A2B9E011AD4253AC38CAFE0A94F704, vertexName=E4FC9D14B84F442E980726892E2118DB, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:32,413 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=C2A2B9E011AD4253AC38CAFE0A94F704, vertexId=vertex_1452791792238_0001_1_01 [E4FC9D14B84F442E980726892E2118DB]
2016-01-14 17:16:32,413 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:32,414 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 3AFBE761E5384F56913198534B38AD75 on vertex vertex_1452791792238_0001_1_00 [E945B299A598457E82C6D04B580E094B]
2016-01-14 17:16:32,415 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:32,415 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791792238_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E4FC9D14B84F442E980726892E2118DB, vertexId=vertex_1452791792238_0001_1_01, initRequestedTime=1452791792405, initedTime=1452791792415, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:32,415 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791792238_0001_1_01 [E4FC9D14B84F442E980726892E2118DB] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:32,416 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791792238_0001_1_00 [E945B299A598457E82C6D04B580E094B] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:32,416 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {3AFBE761E5384F56913198534B38AD75=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:32,416 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791792238_0001_1_00 [E945B299A598457E82C6D04B580E094B] parallelism set to 1
2016-01-14 17:16:32,416 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791792238_0001_1_00 [E945B299A598457E82C6D04B580E094B] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:32,417 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791792238_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791792238_0001_1_00, eventCount=1
2016-01-14 17:16:32,417 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:32,418 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791792238_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E945B299A598457E82C6D04B580E094B, vertexId=vertex_1452791792238_0001_1_00, initRequestedTime=1452791792404, initedTime=1452791792417, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:32,418 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791792238_0001_1_00 [E945B299A598457E82C6D04B580E094B] with distanceFromRoot: 0
2016-01-14 17:16:32,418 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791792238_0001_1_00 [E945B299A598457E82C6D04B580E094B] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:32,418 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in E945B299A598457E82C6D04B580E094B
2016-01-14 17:16:32,418 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791792238_0001_1_00 [E945B299A598457E82C6D04B580E094B]
2016-01-14 17:16:32,419 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:32,419 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791792238_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791792238_0001_1_00, startRequestedTime=1452791792418, startedTime=1452791792418
2016-01-14 17:16:32,419 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791792238_0001_1_00 [E945B299A598457E82C6D04B580E094B] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:32,420 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:32,420 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791792238_0001_1][Event:TASK_STARTED]: vertexName=E945B299A598457E82C6D04B580E094B, taskId=task_1452791792238_0001_1_00_000000, scheduledTime=1452791792420, launchTime=1452791792420
2016-01-14 17:16:32,420 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791792238_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:32,421 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791792238_0001_1_00 for vertex: vertex_1452791792238_0001_1_01 [E4FC9D14B84F442E980726892E2118DB] numStartedSources: 1 numSources: 1
2016-01-14 17:16:32,421 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791792238_0001_1_01 [E4FC9D14B84F442E980726892E2118DB] with distanceFromRoot: 1
2016-01-14 17:16:32,421 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791792238_0001_1_00 [E945B299A598457E82C6D04B580E094B]
2016-01-14 17:16:32,422 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: E945B299A598457E82C6D04B580E094B in vertex: E4FC9D14B84F442E980726892E2118DB
2016-01-14 17:16:32,422 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: E4FC9D14B84F442E980726892E2118DB with 1 source tasks and 1 pending tasks
2016-01-14 17:16:32,422 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:32,422 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791792238_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791792238_0001_1_01, startRequestedTime=1452791792421, startedTime=1452791792421
2016-01-14 17:16:32,423 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791792238_0001_1_01 [E4FC9D14B84F442E980726892E2118DB] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:32,423 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [863904F52F1C4D1DAF49D4E2E6A218CB/F617B39D5A2B426C88C1A640E91E9E31] every/(1/1) ...mblyplatform/-1_ARGS_SWAP, VertexName: E945B299A598457E82C6D04B580E094B, VertexParallelism: 1, TaskAttemptID:attempt_1452791792238_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=3AFBE761E5384F56913198534B38AD75, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E4FC9D14B84F442E980726892E2118DB, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:32,423 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791792238_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:32,423 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:32,426 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:32,426 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:32,426 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:32,426 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791792238_0001_00_000001
2016-01-14 17:16:32,426 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791792238_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:32,428 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791792238_0001_00_000001
2016-01-14 17:16:32,428 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:32,429 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:32,429 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791792238_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791792238_0001_00_000001, launchTime=1452791792429
2016-01-14 17:16:32,430 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791792238_0001_00_000001
2016-01-14 17:16:32,430 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791792238_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:32,434 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791792238_0001, with dag id: dag_1452791792238_0001_1
2016-01-14 17:16:32,441 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791792238_0001_1_00_000000_0] to container: [container_1452791792238_0001_00_000001]
2016-01-14 17:16:32,441 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791792238_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:32,441 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791792238_0001_00_000001 given task: attempt_1452791792238_0001_1_00_000000_0
2016-01-14 17:16:32,441 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791792238_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791792238_0001_1_00_000000_0
2016-01-14 17:16:32,441 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791792238_0001_1_00_000000_0] started. Is using containerId: [container_1452791792238_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:32,441 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791792238_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E945B299A598457E82C6D04B580E094B, taskAttemptId=attempt_1452791792238_0001_1_00_000000_0, startTime=1452791792441, containerId=container_1452791792238_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791792238_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:32,442 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791792238_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:32,442 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:32,442 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791792238_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:32 Starting to run new task attempt: attempt_1452791792238_0001_1_00_000000_0
2016-01-14 17:16:32,442 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791792238_0001_1_00_000000_0
2016-01-14 17:16:32,442 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:32,442 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:32,442 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:32,443 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:32,443 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:32,443 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:32,443 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [863904F52F1C4D1DAF49D4E2E6A218CB/F617B39D5A2B426C88C1A640E91E9E31] every/(1/1) ...mblyplatform/-1_ARGS_SWAP, VertexName: E945B299A598457E82C6D04B580E094B, VertexParallelism: 1, TaskAttemptID:attempt_1452791792238_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=3AFBE761E5384F56913198534B38AD75, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E4FC9D14B84F442E980726892E2118DB, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:32,443 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=764411904
2016-01-14 17:16:32,444 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791792238_0001_1_00_000000_0
2016-01-14 17:16:32,444 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:32,444 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:32,445 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791792238_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:32,445 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:32,445 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=E4FC9D14B84F442E980726892E2118DB, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:32,445 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:32,445 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: E4FC9D14B84F442E980726892E2118DB
2016-01-14 17:16:32,446 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=3AFBE761E5384F56913198534B38AD75, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:32,446 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:32,447 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 3AFBE761E5384F56913198534B38AD75
2016-01-14 17:16:32,447 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 3AFBE761E5384F56913198534B38AD75
2016-01-14 17:16:32,447 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:32,447 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:32,448 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: E4FC9D14B84F442E980726892E2118DB
2016-01-14 17:16:32,448 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: E4FC9D14B84F442E980726892E2118DB
2016-01-14 17:16:32,451 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:32,451 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 3AFBE761E5384F56913198534B38AD75
2016-01-14 17:16:32,452 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 3AFBE761E5384F56913198534B38AD75
2016-01-14 17:16:32,452 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 17:16:32,452 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 17:16:32,454 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E945B299A598457E82C6D04B580E094B, ordinal: 0
2016-01-14 17:16:32,454 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E945B299A598457E82C6D04B580E094B, mem on start (mb), free: 294, total: 729, max: 729
2016-01-14 17:16:32,454 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:32,454 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:32,454 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:32,455 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:32,455 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:32,455 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:32,455 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:32,455 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:32,455 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:32,456 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 764411904, TotalAvailable: 512155975, TotalRequested/TotalJVMHeap:0.14
2016-01-14 17:16:32,456 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 3AFBE761E5384F56913198534B38AD75, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:32,456 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, E4FC9D14B84F442E980726892E2118DB, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:32,456 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:32,456 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 3AFBE761E5384F56913198534B38AD75 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:32,456 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:32,457 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:32,456 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 3AFBE761E5384F56913198534B38AD75
2016-01-14 17:16:32,457 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:32,457 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 3AFBE761E5384F56913198534B38AD75
2016-01-14 17:16:32,457 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:32,458 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791792238_0001_1_00_000000_0
2016-01-14 17:16:32,460 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:32,461 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@612baafc
2016-01-14 17:16:32,461 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:32,462 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 3AFBE761E5384F56913198534B38AD75
2016-01-14 17:16:32,462 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 1B6ADAD1E4A548AABA2ED98DA488A8A9
2016-01-14 17:16:32,463 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 1B6ADAD1E4A548AABA2ED98DA488A8A9
2016-01-14 17:16:32,463 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@612baafc
2016-01-14 17:16:32,463 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:32,464 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:32,463 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@6d67e8d4; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@40e6ee69; comparator=cascading.tuple.hadoop.util.TupleComparator@792e50c8; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:32,465 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:32,465 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:32,465 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:32,491 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:32,491 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:32,492 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:32,492 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:32,492 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:32,493 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:32,494 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:32,494 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E945B299A598457E82C6D04B580E094B, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:32,497 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791792238_0001_1_00_000000_0
2016-01-14 17:16:32,497 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E945B299A598457E82C6D04B580E094B, mem on close (mb), free: 393, total: 729, max: 729
2016-01-14 17:16:32,497 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:32,497 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:32,498 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:32,498 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:32,500 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:32 Completed running task attempt: attempt_1452791792238_0001_1_00_000000_0
2016-01-14 17:16:32,504 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791792238_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:32,505 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791792238_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791792238_0001_1_00, eventCount=1
2016-01-14 17:16:32,506 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791792238_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E945B299A598457E82C6D04B580E094B, taskAttemptId=attempt_1452791792238_0001_1_00_000000_0, startTime=1452791792441, finishTime=1452791792506, timeTaken=65, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=11, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=764411904, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791792494, Process_Duration=2, Process_End_Time=1452791792496, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:32,506 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791792238_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:32,506 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791792238_0001_1_00_000000_0
2016-01-14 17:16:32,506 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:32,507 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:32,507 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:32,507 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791792238_0001_1][Event:TASK_FINISHED]: vertexName=E945B299A598457E82C6D04B580E094B, taskId=task_1452791792238_0001_1_00_000000, startTime=1452791792441, finishTime=1452791792506, timeTaken=65, status=SUCCEEDED, successfulAttemptID=attempt_1452791792238_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=11, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=764411904, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791792494, Process_Duration=2, Process_End_Time=1452791792496, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:32,507 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791792238_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:32,507 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791792238_0001_1_00 [E945B299A598457E82C6D04B580E094B] : 1
2016-01-14 17:16:32,507 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791792238_0001_1_00 [E945B299A598457E82C6D04B580E094B]
2016-01-14 17:16:32,508 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:32,508 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791792238_0001_1][Event:VERTEX_FINISHED]: vertexName=E945B299A598457E82C6D04B580E094B, vertexId=vertex_1452791792238_0001_1_00, initRequestedTime=1452791792404, initedTime=1452791792417, startRequestedTime=1452791792418, startedTime=1452791792418, finishTime=1452791792507, timeTaken=89, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=11, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=764411904, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791792494, Process_Duration=2, Process_End_Time=1452791792496, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791792441, firstTasksToStart=[ task_1452791792238_0001_1_00_000000 ], lastTaskFinishTime=1452791792506, lastTasksToFinish=[ task_1452791792238_0001_1_00_000000 ], minTaskDuration=65, maxTaskDuration=65, avgTaskDuration=65.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791792238_0001_1_00_000000 ], longestDurationTasks=[ task_1452791792238_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:32,508 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791792238_0001_1_00 [E945B299A598457E82C6D04B580E094B] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:32,509 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791792238_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:32,509 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=11, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=764411904, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791792494, Process_Duration=2, Process_End_Time=1452791792496, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:32,509 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791792238_0001_1_01 [E4FC9D14B84F442E980726892E2118DB] attempt: attempt_1452791792238_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:32,509 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:32,509 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:32,510 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: E4FC9D14B84F442E980726892E2118DB
2016-01-14 17:16:32,510 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791792238_0001_1_01 [E4FC9D14B84F442E980726892E2118DB]
2016-01-14 17:16:32,510 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791792238_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791792238_0001_00_000001, stoppedTime=1452791792510, exitStatus=0
2016-01-14 17:16:32,510 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791792238_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:32,510 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791792238_0001_1_00 [E945B299A598457E82C6D04B580E094B] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:32,511 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791792238_0001_1][Event:TASK_STARTED]: vertexName=E4FC9D14B84F442E980726892E2118DB, taskId=task_1452791792238_0001_1_01_000000, scheduledTime=1452791792510, launchTime=1452791792510
2016-01-14 17:16:32,511 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791792238_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:32,511 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:32,511 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:32,511 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791792238_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:32,511 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791792238_0001_1_01 [E4FC9D14B84F442E980726892E2118DB]
2016-01-14 17:16:32,511 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791792238_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:32,512 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [863904F52F1C4D1DAF49D4E2E6A218CB/F617B39D5A2B426C88C1A640E91E9E31] every/(1/1) ...mblyplatform/-1_ARGS_SWAP, VertexName: E4FC9D14B84F442E980726892E2118DB, VertexParallelism: 1, TaskAttemptID:attempt_1452791792238_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E945B299A598457E82C6D04B580E094B, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=C2A2B9E011AD4253AC38CAFE0A94F704, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:32,512 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791792238_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:32,512 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:32,512 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791792238_0001_00_000002
2016-01-14 17:16:32,513 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791792238_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:32,513 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791792238_0001_00_000001
2016-01-14 17:16:32,513 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791792238_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:32,514 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791792238_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:32,514 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791792238_0001_00_000001, Exiting
2016-01-14 17:16:32,514 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791792238_0001_00_000002
2016-01-14 17:16:32,514 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:32,514 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791792238_0001_00_000001
2016-01-14 17:16:32,515 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791792238_0001_00_000001
2016-01-14 17:16:32,516 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:32,516 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791792238_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:32,516 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791792238_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791792238_0001_00_000002, launchTime=1452791792515
2016-01-14 17:16:32,517 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791792238_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:32,517 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791792238_0001_00_000001 completed successfully
2016-01-14 17:16:32,517 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791792238_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:32,518 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791792238_0001_00_000002
2016-01-14 17:16:32,518 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791792238_0001_1_01_000000_0] to container: [container_1452791792238_0001_00_000002]
2016-01-14 17:16:32,518 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791792238_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:32,519 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791792238_0001_1_01_000000_0] started. Is using containerId: [container_1452791792238_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:32,519 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791792238_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E4FC9D14B84F442E980726892E2118DB, taskAttemptId=attempt_1452791792238_0001_1_01_000000_0, startTime=1452791792519, containerId=container_1452791792238_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791792238_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:32,519 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:32,519 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791792238_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:32,519 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791792238_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:32,519 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791792238_0001_00_000002 given task: attempt_1452791792238_0001_1_01_000000_0
2016-01-14 17:16:32,519 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791792238_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791792238_0001_1_01_000000_0
2016-01-14 17:16:32 Starting to run new task attempt: attempt_1452791792238_0001_1_01_000000_0
2016-01-14 17:16:32,520 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791792238_0001_1_01_000000_0
2016-01-14 17:16:32,520 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:32,520 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:32,520 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:32,520 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:32,521 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:32,521 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:32,521 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [863904F52F1C4D1DAF49D4E2E6A218CB/F617B39D5A2B426C88C1A640E91E9E31] every/(1/1) ...mblyplatform/-1_ARGS_SWAP, VertexName: E4FC9D14B84F442E980726892E2118DB, VertexParallelism: 1, TaskAttemptID:attempt_1452791792238_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E945B299A598457E82C6D04B580E094B, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=C2A2B9E011AD4253AC38CAFE0A94F704, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:32,521 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=764411904
2016-01-14 17:16:32,521 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791792238_0001_1_01_000000_0
2016-01-14 17:16:32,521 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:32,521 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:32,522 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=E945B299A598457E82C6D04B580E094B, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:32,522 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:32,522 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: E945B299A598457E82C6D04B580E094B
2016-01-14 17:16:32,523 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 687970688, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:32,524 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:32,524 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 687970688, type: INPUT, componentVertexName: E945B299A598457E82C6D04B580E094B
2016-01-14 17:16:32,524 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: E945B299A598457E82C6D04B580E094B
2016-01-14 17:16:32,524 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:32,524 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791792238_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:32,525 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=C2A2B9E011AD4253AC38CAFE0A94F704, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:32,526 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:32,526 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: C2A2B9E011AD4253AC38CAFE0A94F704
2016-01-14 17:16:32,526 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:32,526 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:32,527 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: C2A2B9E011AD4253AC38CAFE0A94F704
2016-01-14 17:16:32,532 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E4FC9D14B84F442E980726892E2118DB, ordinal: 1
2016-01-14 17:16:32,532 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E4FC9D14B84F442E980726892E2118DB, mem on start (mb), free: 388, total: 729, max: 729
2016-01-14 17:16:32,532 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:32,532 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:32,533 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:32,538 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:32,538 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: C2A2B9E011AD4253AC38CAFE0A94F704
2016-01-14 17:16:32,539 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:32,539 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:32,539 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:32,539 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:32,539 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:32,539 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:32,539 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 687970688, TotalRequestedScaled: 6.350498658461539E8, TotalJVMHeap: 764411904, TotalAvailable: 512155975, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:32,540 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, E945B299A598457E82C6D04B580E094B, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=687970688, allocated=512155975
2016-01-14 17:16:32,540 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, C2A2B9E011AD4253AC38CAFE0A94F704, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:32,540 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:32,540 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: E945B299A598457E82C6D04B580E094B being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:32,540 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:32,540 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: E945B299A598457E82C6D04B580E094B
2016-01-14 17:16:32,540 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:32,541 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:32,541 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:32,541 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: E945B299A598457E82C6D04B580E094B with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:32,541 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=687970688, postMergeMem=0, RuntimeTotalAvailable=512155975. Updated to: ShuffleMem=512155975, postMergeMem: 0
2016-01-14 17:16:32,541 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=512155975, maxSingleShuffleLimit=128038992, mergeThreshold=460940352, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:32,542 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:32,542 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: E945B299A598457E82C6D04B580E094B
2016-01-14 17:16:32,543 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:32,543 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791792238_0001_1_01_000000_0
2016-01-14 17:16:32,545 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791792238_0001_1_00_000000_0_10110, runDuration: 56854, ]
2016-01-14 17:16:32,546 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:32,546 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791792238_0001_1_00_000000_0_10110] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791792546, TimeTaken=0, Rate=0.00 MB/s
2016-01-14 17:16:32,546 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : E945B299A598457E82C6D04B580E094B
2016-01-14 17:16:32,547 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:32,547 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 1B6ADAD1E4A548AABA2ED98DA488A8A9
2016-01-14 17:16:32,547 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [E945B299A598457E82C6D04B580E094B] #19 in 2ms
2016-01-14 17:16:32,547 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_ARGS_SWAP"], id: C2A2B9E011AD4253AC38CAFE0A94F704
2016-01-14 17:16:32,547 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [E945B299A598457E82C6D04B580E094B] #19, status:false, isInterrupted:false
2016-01-14 17:16:32,547 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_ARGS_SWAP"]
2016-01-14 17:16:32,548 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:32,548 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 1B6ADAD1E4A548AABA2ED98DA488A8A9, for 1 inputs
2016-01-14 17:16:32,548 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:32,550 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:32,550 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:32,550 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:32,550 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : E945B299A598457E82C6D04B580E094B
2016-01-14 17:16:32,550 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:32,551 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E4FC9D14B84F442E980726892E2118DB, all 1 inputs ready in: 00:00:00.001
2016-01-14 17:16:32,552 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@5afcc97e; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@a109ddd
2016-01-14 17:16:32,557 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:32,558 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:32,558 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791792238_0001_1_01_000000_0
2016-01-14 17:16:32,558 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791792238_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:32,559 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791792238_0001_1_01_000000_0
2016-01-14 17:16:32,559 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E4FC9D14B84F442E980726892E2118DB, mem on close (mb), free: 385, total: 729, max: 729
2016-01-14 17:16:32,559 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: E945B299A598457E82C6D04B580E094B
2016-01-14 17:16:32,559 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:32,559 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:32,560 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:32,560 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:32,560 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:32 Completed running task attempt: attempt_1452791792238_0001_1_01_000000_0
2016-01-14 17:16:32,560 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791792238_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:32,561 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=187, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=764411904, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791792551, Process_Duration=8, Process_End_Time=1452791792559, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 17:16:32,561 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:32,561 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791792238_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E4FC9D14B84F442E980726892E2118DB, taskAttemptId=attempt_1452791792238_0001_1_01_000000_0, startTime=1452791792519, finishTime=1452791792561, timeTaken=42, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=187, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=764411904, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791792551, Process_Duration=8, Process_End_Time=1452791792559, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:32,561 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:32,562 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:32,562 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791792238_0001_00_000002
2016-01-14 17:16:32,562 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791792238_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:32,562 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:32,562 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791792238_0001_1_01_000000_0
2016-01-14 17:16:32,563 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:32,563 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791792238_0001_1][Event:TASK_FINISHED]: vertexName=E4FC9D14B84F442E980726892E2118DB, taskId=task_1452791792238_0001_1_01_000000, startTime=1452791792519, finishTime=1452791792562, timeTaken=43, status=SUCCEEDED, successfulAttemptID=attempt_1452791792238_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=187, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=764411904, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791792551, Process_Duration=8, Process_End_Time=1452791792559, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:32,563 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791792238_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:32,563 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791792238_0001_1_01 [E4FC9D14B84F442E980726892E2118DB] : 1
2016-01-14 17:16:32,563 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791792238_0001_1_01 [E4FC9D14B84F442E980726892E2118DB]
2016-01-14 17:16:32,564 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:32,564 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791792238_0001_1][Event:VERTEX_FINISHED]: vertexName=E4FC9D14B84F442E980726892E2118DB, vertexId=vertex_1452791792238_0001_1_01, initRequestedTime=1452791792405, initedTime=1452791792415, startRequestedTime=1452791792421, startedTime=1452791792421, finishTime=1452791792563, timeTaken=142, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=187, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=764411904, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791792551, Process_Duration=8, Process_End_Time=1452791792559, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452791792519, firstTasksToStart=[ task_1452791792238_0001_1_01_000000 ], lastTaskFinishTime=1452791792561, lastTasksToFinish=[ task_1452791792238_0001_1_01_000000 ], minTaskDuration=42, maxTaskDuration=42, avgTaskDuration=42.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791792238_0001_1_01_000000 ], longestDurationTasks=[ task_1452791792238_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:32,564 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791792238_0001_1_01 [E4FC9D14B84F442E980726892E2118DB] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:32,564 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791792238_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:32,565 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791792238_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791792238_0001_00_000002, stoppedTime=1452791792565, exitStatus=0
2016-01-14 17:16:32,565 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791792238_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:32,565 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791792238_0001_1_01 [E4FC9D14B84F442E980726892E2118DB] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:32,565 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791792238_0001_1
2016-01-14 17:16:32,565 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791792238_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452791792238_0001_1
2016-01-14 17:16:32,565 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:32,566 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: C2A2B9E011AD4253AC38CAFE0A94F704 for vertex: vertex_1452791792238_0001_1_01 [E4FC9D14B84F442E980726892E2118DB]
2016-01-14 17:16:32,574 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791792238_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 17:16:32,574 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791792238_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:32,574 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791792238_0001_00_000002, Exiting
2016-01-14 17:16:32,574 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791792238_0001_00_000002
2016-01-14 17:16:32,574 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791792238_0001_00_000002
2016-01-14 17:16:32,574 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791792238_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:32,574 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791792238_0001_00_000002 completed successfully
2016-01-14 17:16:32,575 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452791792238_0001_1_00 [E945B299A598457E82C6D04B580E094B]
2016-01-14 17:16:32,576 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791792238_0001_1, queueSize=0
2016-01-14 17:16:32,577 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:32,577 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791792238_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791792238_0001_1, startTime=1452791792403, finishTime=1452791792575, timeTaken=172, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=520, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=11, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1528823808, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905583585045, Process_Duration=10, Process_End_Time=2905583585055, Read_Duration=4, Tuples_Read=40, Tuples_Written=40, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 17:16:32,577 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791792238_0001_1 finished with state: SUCCEEDED
2016-01-14 17:16:32,577 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791792238_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:32,578 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791792238_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:16:32,578 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452791792238_0001_00_000002
2016-01-14 17:16:32 Completed Dag: dag_1452791792238_0001_1
2016-01-14 17:16:32,578 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791792238_0001_1, dagState=SUCCEEDED
2016-01-14 17:16:32,578 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791792238_0001_1_post
2016-01-14 17:16:32,578 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:32,579 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:32,585 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...mblyplatform/-1_ARGS_SWAP, applicationId=application_1452791792238_0001
2016-01-14 17:16:32,587 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:32,587 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:32,587 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:32,587 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:32,587 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:32,587 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:32,588 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:32,588 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:32,588 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:32,589 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:32,589 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:32,589 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:32,590 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:32,594 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/863904F52F1C4D1DAF49D4E2E6A218CB/F617B39D5A2B426C88C1A640E91E9E31/.tez/application_1452791792238_0001
2016-01-14 17:16:32,595 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.374
2016-01-14 17:16:32,660 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_ARGS_SWAP
2016-01-14 17:16:32,681 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:32,682 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:32,713 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.007
2016-01-14 17:16:32,721 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-01-14 17:16:32,722 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:32,722 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:32,722 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:32,736 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:32,742 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_-1
2016-01-14 17:16:32,788 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:32,788 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:32,788 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_-1"]
2016-01-14 17:16:32,788 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:32,788 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:32,788 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:32,789 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...mblyplatform/0_'field'_-1
2016-01-14 17:16:32,789 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:32,800 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:32,805 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:32,805 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:32,805 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:32,816 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/83CDB518F31D4E4E90AA92A69D1C73B0/C8AC0F1F0F8C4AE98D272658EEFEA3BE/.tez/application_1452791792805_0001 doesn't exist and is created
2016-01-14 17:16:32,831 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:32,831 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:32,832 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/83CDB518F31D4E4E90AA92A69D1C73B0/C8AC0F1F0F8C4AE98D272658EEFEA3BE/.tez/application_1452791792805_0001
2016-01-14 17:16:32,835 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791792805_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:32,846 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:32,846 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:32,848 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:32,848 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:32,849 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/83CDB518F31D4E4E90AA92A69D1C73B0/C8AC0F1F0F8C4AE98D272658EEFEA3BE/.tez/application_1452791792805_0001/localmode-log-dir
2016-01-14 17:16:32,849 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/83CDB518F31D4E4E90AA92A69D1C73B0/C8AC0F1F0F8C4AE98D272658EEFEA3BE/.tez/application_1452791792805_0001/localmode-log-dir/history.txt.appattempt_1452791792805_0001_000000, maxErrors=10
2016-01-14 17:16:32,849 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:32,849 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791792805_0001_000000, appSubmitTime=1452791792835, launchTime=1452791792835
2016-01-14 17:16:32,855 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:59072
2016-01-14 17:16:32,855 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:32,860 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:32,861 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:32,861 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:32,862 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:32,865 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791792805_0001_000000, startTime=1452791792865
2016-01-14 17:16:32,865 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:32,865 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:32,932 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:32,932 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:32,932 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...mblyplatform/0_'field'_-1, applicationId=application_1452791792805_0001, dagName=[83CDB518F31D4E4E90AA92A69D1C73B0/C8AC0F1F0F8C4AE98D272658EEFEA3BE] every/(1/1) ...mblyplatform/0_'field'_-1
2016-01-14 17:16:32,946 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [83CDB518F31D4E4E90AA92A69D1C73B0/C8AC0F1F0F8C4AE98D272658EEFEA3BE] every/(1/1) ...mblyplatform/0_'field'_-1
2016-01-14 17:16:32,948 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791792805_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/83CDB518F31D4E4E90AA92A69D1C73B0/C8AC0F1F0F8C4AE98D272658EEFEA3BE/.tez/application_1452791792805_0001/localmode-log-dir/dag_1452791792805_0001_1.dot
2016-01-14 17:16:32,949 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/83CDB518F31D4E4E90AA92A69D1C73B0/C8AC0F1F0F8C4AE98D272658EEFEA3BE/.tez/application_1452791792805_0001/localmode-log-dir/dag_1452791792805_0001_1-tez-dag.pb.txt
2016-01-14 17:16:32,955 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791792805_0001_1
2016-01-14 17:16:32,956 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:32,956 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [83CDB518F31D4E4E90AA92A69D1C73B0/C8AC0F1F0F8C4AE98D272658EEFEA3BE] every/(1/1) ...mblyplatform/0_'field'_-1
2016-01-14 17:16:32 Running Dag: dag_1452791792805_0001_1
2016-01-14 17:16:32,969 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:32,969 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791792805_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791792805_0001_1, submitTime=1452791792946
2016-01-14 17:16:32,970 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex FA939928E4314454B93A512D2686548D
2016-01-14 17:16:32,971 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:32,972 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791792805_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791792805_0001_1, initTime=1452791792969
2016-01-14 17:16:32,972 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:32,972 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791792805_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:32,973 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791792805_0001_1][Event:DAG_STARTED]: dagID=dag_1452791792805_0001_1, startTime=1452791792972
2016-01-14 17:16:32,973 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:32,973 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791792805_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:32,973 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: ECCCCDF20D0A466A99C7A41645DB23AE : {5DE343215B684250A6207A0A5CE890C2={InputName=5DE343215B684250A6207A0A5CE890C2}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:32,973 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 5DE343215B684250A6207A0A5CE890C2, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:32,974 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791792805_0001_1_00 [ECCCCDF20D0A466A99C7A41645DB23AE]
2016-01-14 17:16:32,974 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791792805_0001_1_00 [ECCCCDF20D0A466A99C7A41645DB23AE]
2016-01-14 17:16:32,974 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791792805_0001_1_00 [ECCCCDF20D0A466A99C7A41645DB23AE]
2016-01-14 17:16:32,974 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791792805_0001_1_00 [ECCCCDF20D0A466A99C7A41645DB23AE]. Starting root input initializers: 1
2016-01-14 17:16:32,974 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...mblyplatform/0_'field'_-1, applicationId=application_1452791792805_0001, dagName=[83CDB518F31D4E4E90AA92A69D1C73B0/C8AC0F1F0F8C4AE98D272658EEFEA3BE] every/(1/1) ...mblyplatform/0_'field'_-1
2016-01-14 17:16:32,975 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 5DE343215B684250A6207A0A5CE890C2 on vertex vertex_1452791792805_0001_1_00 [ECCCCDF20D0A466A99C7A41645DB23AE]
2016-01-14 17:16:32,973 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:32,974 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791792805_0001_1_00 [ECCCCDF20D0A466A99C7A41645DB23AE] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:32,976 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791792805_0001_1_01 [FA939928E4314454B93A512D2686548D]
2016-01-14 17:16:32,977 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:32,978 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 5DE343215B684250A6207A0A5CE890C2 asking for 1 tasks. Headroom: 729 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:32,978 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:32,979 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:32,979 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:32,979 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791792805_0001_1_01 [FA939928E4314454B93A512D2686548D]
2016-01-14 17:16:32,979 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791792805_0001_1_01 [FA939928E4314454B93A512D2686548D]
2016-01-14 17:16:32,980 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791792805_0001_1_01 [FA939928E4314454B93A512D2686548D]
2016-01-14 17:16:32,980 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=E7491985CB124B128CFECEDD4DFBD50B, vertexId=vertex_1452791792805_0001_1_01 [FA939928E4314454B93A512D2686548D], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:32,980 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=E7491985CB124B128CFECEDD4DFBD50B, vertexId=vertex_1452791792805_0001_1_01 [FA939928E4314454B93A512D2686548D]
2016-01-14 17:16:32,981 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:32,982 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 5DE343215B684250A6207A0A5CE890C2 on vertex vertex_1452791792805_0001_1_00 [ECCCCDF20D0A466A99C7A41645DB23AE]
2016-01-14 17:16:32,982 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=E7491985CB124B128CFECEDD4DFBD50B, vertexName=FA939928E4314454B93A512D2686548D, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:32,982 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=E7491985CB124B128CFECEDD4DFBD50B, vertexId=vertex_1452791792805_0001_1_01 [FA939928E4314454B93A512D2686548D]
2016-01-14 17:16:32,984 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:32,984 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791792805_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FA939928E4314454B93A512D2686548D, vertexId=vertex_1452791792805_0001_1_01, initRequestedTime=1452791792976, initedTime=1452791792984, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:32,984 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791792805_0001_1_01 [FA939928E4314454B93A512D2686548D] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:32,984 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791792805_0001_1_00 [ECCCCDF20D0A466A99C7A41645DB23AE] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:32,984 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {5DE343215B684250A6207A0A5CE890C2=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:32,985 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791792805_0001_1_00 [ECCCCDF20D0A466A99C7A41645DB23AE] parallelism set to 1
2016-01-14 17:16:32,985 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791792805_0001_1_00 [ECCCCDF20D0A466A99C7A41645DB23AE] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:32,985 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791792805_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791792805_0001_1_00, eventCount=1
2016-01-14 17:16:32,985 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:32,986 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791792805_0001_1][Event:VERTEX_INITIALIZED]: vertexName=ECCCCDF20D0A466A99C7A41645DB23AE, vertexId=vertex_1452791792805_0001_1_00, initRequestedTime=1452791792973, initedTime=1452791792985, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:32,986 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791792805_0001_1_00 [ECCCCDF20D0A466A99C7A41645DB23AE] with distanceFromRoot: 0
2016-01-14 17:16:32,986 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791792805_0001_1_00 [ECCCCDF20D0A466A99C7A41645DB23AE] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:32,986 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in ECCCCDF20D0A466A99C7A41645DB23AE
2016-01-14 17:16:32,986 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791792805_0001_1_00 [ECCCCDF20D0A466A99C7A41645DB23AE]
2016-01-14 17:16:32,987 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:32,987 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791792805_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791792805_0001_1_00, startRequestedTime=1452791792986, startedTime=1452791792986
2016-01-14 17:16:32,987 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791792805_0001_1_00 [ECCCCDF20D0A466A99C7A41645DB23AE] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:32,988 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:32,988 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791792805_0001_1][Event:TASK_STARTED]: vertexName=ECCCCDF20D0A466A99C7A41645DB23AE, taskId=task_1452791792805_0001_1_00_000000, scheduledTime=1452791792988, launchTime=1452791792988
2016-01-14 17:16:32,988 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791792805_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:32,989 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791792805_0001_1_00 for vertex: vertex_1452791792805_0001_1_01 [FA939928E4314454B93A512D2686548D] numStartedSources: 1 numSources: 1
2016-01-14 17:16:32,989 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791792805_0001_1_01 [FA939928E4314454B93A512D2686548D] with distanceFromRoot: 1
2016-01-14 17:16:32,989 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791792805_0001_1_00 [ECCCCDF20D0A466A99C7A41645DB23AE]
2016-01-14 17:16:32,990 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: ECCCCDF20D0A466A99C7A41645DB23AE in vertex: FA939928E4314454B93A512D2686548D
2016-01-14 17:16:32,990 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: FA939928E4314454B93A512D2686548D with 1 source tasks and 1 pending tasks
2016-01-14 17:16:32,990 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:32,990 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791792805_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791792805_0001_1_01, startRequestedTime=1452791792989, startedTime=1452791792989
2016-01-14 17:16:32,990 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791792805_0001_1_01 [FA939928E4314454B93A512D2686548D] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:32,991 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [83CDB518F31D4E4E90AA92A69D1C73B0/C8AC0F1F0F8C4AE98D272658EEFEA3BE] every/(1/1) ...mblyplatform/0_'field'_-1, VertexName: ECCCCDF20D0A466A99C7A41645DB23AE, VertexParallelism: 1, TaskAttemptID:attempt_1452791792805_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=5DE343215B684250A6207A0A5CE890C2, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FA939928E4314454B93A512D2686548D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:32,991 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791792805_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:32,991 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:32,992 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:32,992 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:32,992 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:32,992 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791792805_0001_00_000001
2016-01-14 17:16:32,992 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791792805_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:32,994 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791792805_0001_00_000001
2016-01-14 17:16:32,994 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:32,994 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791792805_0001, with dag id: dag_1452791792805_0001_1
2016-01-14 17:16:32,995 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791792805_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:32,996 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791792805_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791792805_0001_00_000001, launchTime=1452791792995
2016-01-14 17:16:32,996 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:32,998 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791792805_0001_00_000001
2016-01-14 17:16:32,998 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791792805_0001_1_00_000000_0] to container: [container_1452791792805_0001_00_000001]
2016-01-14 17:16:32,999 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791792805_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:32,999 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791792805_0001_00_000001 given task: attempt_1452791792805_0001_1_00_000000_0
2016-01-14 17:16:33,000 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791792805_0001_1_00_000000_0] started. Is using containerId: [container_1452791792805_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:33,000 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791792805_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=ECCCCDF20D0A466A99C7A41645DB23AE, taskAttemptId=attempt_1452791792805_0001_1_00_000000_0, startTime=1452791792999, containerId=container_1452791792805_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791792805_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:33,000 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791792805_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:33,000 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791792805_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:33,000 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:33,000 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791792805_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791792805_0001_1_00_000000_0
2016-01-14 17:16:33 Starting to run new task attempt: attempt_1452791792805_0001_1_00_000000_0
2016-01-14 17:16:33,002 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791792805_0001_1_00_000000_0
2016-01-14 17:16:33,003 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:33,003 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:33,003 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:33,004 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:33,004 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:33,004 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:33,004 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [83CDB518F31D4E4E90AA92A69D1C73B0/C8AC0F1F0F8C4AE98D272658EEFEA3BE] every/(1/1) ...mblyplatform/0_'field'_-1, VertexName: ECCCCDF20D0A466A99C7A41645DB23AE, VertexParallelism: 1, TaskAttemptID:attempt_1452791792805_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=5DE343215B684250A6207A0A5CE890C2, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FA939928E4314454B93A512D2686548D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:33,005 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=764411904
2016-01-14 17:16:33,006 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791792805_0001_1_00_000000_0
2016-01-14 17:16:33,006 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:33,006 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:33,007 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791792805_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:33,007 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=5DE343215B684250A6207A0A5CE890C2, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:33,007 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:33,007 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=FA939928E4314454B93A512D2686548D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:33,007 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:33,008 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 5DE343215B684250A6207A0A5CE890C2
2016-01-14 17:16:33,008 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 5DE343215B684250A6207A0A5CE890C2
2016-01-14 17:16:33,008 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:33,009 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: FA939928E4314454B93A512D2686548D
2016-01-14 17:16:33,010 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:33,011 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 5DE343215B684250A6207A0A5CE890C2
2016-01-14 17:16:33,011 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 5DE343215B684250A6207A0A5CE890C2
2016-01-14 17:16:33,011 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:33,011 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: FA939928E4314454B93A512D2686548D
2016-01-14 17:16:33,012 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: FA939928E4314454B93A512D2686548D
2016-01-14 17:16:33,012 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:33,016 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: ECCCCDF20D0A466A99C7A41645DB23AE, ordinal: 0
2016-01-14 17:16:33,016 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: ECCCCDF20D0A466A99C7A41645DB23AE, mem on start (mb), free: 293, total: 729, max: 729
2016-01-14 17:16:33,016 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:33,017 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:33,017 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:33,017 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:33,017 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:33,018 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:33,018 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:33,018 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:33,018 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:33,018 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 764411904, TotalAvailable: 512155975, TotalRequested/TotalJVMHeap:0.14
2016-01-14 17:16:33,019 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 5DE343215B684250A6207A0A5CE890C2, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:33,019 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, FA939928E4314454B93A512D2686548D, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:33,019 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:33,019 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 5DE343215B684250A6207A0A5CE890C2 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:33,019 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:33,019 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:33,019 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 5DE343215B684250A6207A0A5CE890C2
2016-01-14 17:16:33,020 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:33,020 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 5DE343215B684250A6207A0A5CE890C2
2016-01-14 17:16:33,020 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:33,021 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791792805_0001_1_00_000000_0
2016-01-14 17:16:33,021 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:33,021 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@b7ccec1
2016-01-14 17:16:33,022 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@b7ccec1
2016-01-14 17:16:33,022 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:33,022 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:33,024 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:33,024 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 5DE343215B684250A6207A0A5CE890C2
2016-01-14 17:16:33,024 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 40E9C66B2F934CACB830DCAB6001F3CA
2016-01-14 17:16:33,025 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 40E9C66B2F934CACB830DCAB6001F3CA
2016-01-14 17:16:33,025 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@aa0399b; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@1d25f490; comparator=cascading.tuple.hadoop.util.TupleComparator@26db28e3; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:33,025 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:33,026 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:33,026 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:33,051 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:33,051 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:33,052 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:33,052 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:33,052 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:33,053 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:33,053 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:33,053 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: ECCCCDF20D0A466A99C7A41645DB23AE, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:33,056 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791792805_0001_1_00_000000_0
2016-01-14 17:16:33,056 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: ECCCCDF20D0A466A99C7A41645DB23AE, mem on close (mb), free: 390, total: 727, max: 727
2016-01-14 17:16:33,056 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:33,056 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:33,056 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:33,056 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:33,059 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:33 Completed running task attempt: attempt_1452791792805_0001_1_00_000000_0
2016-01-14 17:16:33,062 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791792805_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:33,062 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=12, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=762314752, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791793053, Process_Duration=3, Process_End_Time=1452791793056, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:33,063 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:33,062 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791792805_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791792805_0001_1_00, eventCount=1
2016-01-14 17:16:33,063 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:33,063 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791792805_0001_00_000001
2016-01-14 17:16:33,063 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791792805_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=ECCCCDF20D0A466A99C7A41645DB23AE, taskAttemptId=attempt_1452791792805_0001_1_00_000000_0, startTime=1452791792999, finishTime=1452791793063, timeTaken=64, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=12, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=762314752, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791793053, Process_Duration=3, Process_End_Time=1452791793056, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:33,063 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:33,064 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791792805_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:33,064 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791792805_0001_1_00_000000_0
2016-01-14 17:16:33,064 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:33,064 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791792805_0001_1][Event:TASK_FINISHED]: vertexName=ECCCCDF20D0A466A99C7A41645DB23AE, taskId=task_1452791792805_0001_1_00_000000, startTime=1452791792999, finishTime=1452791793064, timeTaken=65, status=SUCCEEDED, successfulAttemptID=attempt_1452791792805_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=12, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=762314752, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791793053, Process_Duration=3, Process_End_Time=1452791793056, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:33,065 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791792805_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:33,065 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791792805_0001_1_00 [ECCCCDF20D0A466A99C7A41645DB23AE] : 1
2016-01-14 17:16:33,065 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791792805_0001_1_00 [ECCCCDF20D0A466A99C7A41645DB23AE]
2016-01-14 17:16:33,066 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:33,066 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791792805_0001_1][Event:VERTEX_FINISHED]: vertexName=ECCCCDF20D0A466A99C7A41645DB23AE, vertexId=vertex_1452791792805_0001_1_00, initRequestedTime=1452791792973, initedTime=1452791792985, startRequestedTime=1452791792986, startedTime=1452791792986, finishTime=1452791793065, timeTaken=79, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=12, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=762314752, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791793053, Process_Duration=3, Process_End_Time=1452791793056, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791792999, firstTasksToStart=[ task_1452791792805_0001_1_00_000000 ], lastTaskFinishTime=1452791793063, lastTasksToFinish=[ task_1452791792805_0001_1_00_000000 ], minTaskDuration=64, maxTaskDuration=64, avgTaskDuration=64.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791792805_0001_1_00_000000 ], longestDurationTasks=[ task_1452791792805_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:33,066 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791792805_0001_1_00 [ECCCCDF20D0A466A99C7A41645DB23AE] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:33,066 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791792805_0001_1_01 [FA939928E4314454B93A512D2686548D] attempt: attempt_1452791792805_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:33,066 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: FA939928E4314454B93A512D2686548D
2016-01-14 17:16:33,066 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791792805_0001_1_01 [FA939928E4314454B93A512D2686548D]
2016-01-14 17:16:33,067 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791792805_0001_1_00 [ECCCCDF20D0A466A99C7A41645DB23AE] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:33,067 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791792805_0001_1][Event:TASK_STARTED]: vertexName=FA939928E4314454B93A512D2686548D, taskId=task_1452791792805_0001_1_01_000000, scheduledTime=1452791793067, launchTime=1452791793067
2016-01-14 17:16:33,067 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:33,067 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:33,068 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791792805_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:33,069 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791792805_0001_1_01 [FA939928E4314454B93A512D2686548D]
2016-01-14 17:16:33,069 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791792805_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:33,069 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:33,069 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791792805_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791792805_0001_00_000001, stoppedTime=1452791793069, exitStatus=0
2016-01-14 17:16:33,070 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791792805_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:33,070 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [83CDB518F31D4E4E90AA92A69D1C73B0/C8AC0F1F0F8C4AE98D272658EEFEA3BE] every/(1/1) ...mblyplatform/0_'field'_-1, VertexName: FA939928E4314454B93A512D2686548D, VertexParallelism: 1, TaskAttemptID:attempt_1452791792805_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=ECCCCDF20D0A466A99C7A41645DB23AE, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=E7491985CB124B128CFECEDD4DFBD50B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:33,070 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791792805_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:33,070 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791792805_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:33,070 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:33,071 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791792805_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:33,071 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791792805_0001_00_000002
2016-01-14 17:16:33,071 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791792805_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:33,073 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791792805_0001_00_000002
2016-01-14 17:16:33,073 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:33,073 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:33,073 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791792805_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791792805_0001_00_000002, launchTime=1452791793073
2016-01-14 17:16:33,073 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791792805_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:33,074 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791792805_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:33,074 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791792805_0001_00_000002
2016-01-14 17:16:33,074 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791792805_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:33,074 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791792805_0001_00_000001, Exiting
2016-01-14 17:16:33,075 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791792805_0001_00_000001
2016-01-14 17:16:33,075 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791792805_0001_00_000001
2016-01-14 17:16:33,074 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791792805_0001_1_01_000000_0] to container: [container_1452791792805_0001_00_000002]
2016-01-14 17:16:33,075 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791792805_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:33,075 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791792805_0001_00_000002 given task: attempt_1452791792805_0001_1_01_000000_0
2016-01-14 17:16:33,075 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791792805_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791792805_0001_1_01_000000_0
2016-01-14 17:16:33 Starting to run new task attempt: attempt_1452791792805_0001_1_01_000000_0
2016-01-14 17:16:33,076 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791792805_0001_1_01_000000_0
2016-01-14 17:16:33,076 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:33,076 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:33,076 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:33,076 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:33,076 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:33,076 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:33,077 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [83CDB518F31D4E4E90AA92A69D1C73B0/C8AC0F1F0F8C4AE98D272658EEFEA3BE] every/(1/1) ...mblyplatform/0_'field'_-1, VertexName: FA939928E4314454B93A512D2686548D, VertexParallelism: 1, TaskAttemptID:attempt_1452791792805_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=ECCCCDF20D0A466A99C7A41645DB23AE, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=E7491985CB124B128CFECEDD4DFBD50B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:33,077 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791792805_0001_1_01_000000_0] started. Is using containerId: [container_1452791792805_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:33,077 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=762314752
2016-01-14 17:16:33,077 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:33,077 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791792805_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FA939928E4314454B93A512D2686548D, taskAttemptId=attempt_1452791792805_0001_1_01_000000_0, startTime=1452791793075, containerId=container_1452791792805_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791792805_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:33,077 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791792805_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:33,078 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791792805_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:33,078 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791792805_0001_1_01_000000_0
2016-01-14 17:16:33,078 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:33,078 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791792805_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:33,079 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:33,078 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791792805_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:33,079 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791792805_0001_00_000001 completed successfully
2016-01-14 17:16:33,080 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=ECCCCDF20D0A466A99C7A41645DB23AE, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:33,080 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:33,081 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: ECCCCDF20D0A466A99C7A41645DB23AE
2016-01-14 17:16:33,080 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:33,080 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=E7491985CB124B128CFECEDD4DFBD50B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:33,080 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791792805_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:33,082 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:33,082 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: E7491985CB124B128CFECEDD4DFBD50B
2016-01-14 17:16:33,083 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:33,083 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: E7491985CB124B128CFECEDD4DFBD50B
2016-01-14 17:16:33,083 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:33,085 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 686083264, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:33,085 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:33,085 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 686083264, type: INPUT, componentVertexName: ECCCCDF20D0A466A99C7A41645DB23AE
2016-01-14 17:16:33,085 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: ECCCCDF20D0A466A99C7A41645DB23AE
2016-01-14 17:16:33,089 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FA939928E4314454B93A512D2686548D, ordinal: 1
2016-01-14 17:16:33,089 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FA939928E4314454B93A512D2686548D, mem on start (mb), free: 384, total: 727, max: 727
2016-01-14 17:16:33,089 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:33,089 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:33,090 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:33,093 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:33,093 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: E7491985CB124B128CFECEDD4DFBD50B
2016-01-14 17:16:33,093 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:33,094 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:33,094 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:33,094 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:33,094 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:33,094 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:33,094 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 686083264, TotalRequestedScaled: 6.333076283076923E8, TotalJVMHeap: 762314752, TotalAvailable: 510750883, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:33,094 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, E7491985CB124B128CFECEDD4DFBD50B, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:33,095 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, ECCCCDF20D0A466A99C7A41645DB23AE, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=686083264, allocated=510750883
2016-01-14 17:16:33,095 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:33,095 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: ECCCCDF20D0A466A99C7A41645DB23AE being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:33,095 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: ECCCCDF20D0A466A99C7A41645DB23AE
2016-01-14 17:16:33,096 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:33,096 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:33,096 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:33,096 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:33,098 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: ECCCCDF20D0A466A99C7A41645DB23AE with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:33,098 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=686083264, postMergeMem=0, RuntimeTotalAvailable=510750883. Updated to: ShuffleMem=510750883, postMergeMem: 0
2016-01-14 17:16:33,098 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=510750883, maxSingleShuffleLimit=127687720, mergeThreshold=459675776, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:33,098 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:33,099 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: ECCCCDF20D0A466A99C7A41645DB23AE
2016-01-14 17:16:33,099 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:33,102 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791792805_0001_1_01_000000_0
2016-01-14 17:16:33,103 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791792805_0001_1_00_000000_0_10117, runDuration: 51475, ]
2016-01-14 17:16:33,106 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:33,107 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 40E9C66B2F934CACB830DCAB6001F3CA
2016-01-14 17:16:33,107 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_-1"], id: E7491985CB124B128CFECEDD4DFBD50B
2016-01-14 17:16:33,107 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791792805_0001_1_00_000000_0_10117] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791793107, TimeTaken=0, Rate=0.00 MB/s
2016-01-14 17:16:33,107 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : ECCCCDF20D0A466A99C7A41645DB23AE
2016-01-14 17:16:33,107 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_-1"]
2016-01-14 17:16:33,108 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:33,108 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 40E9C66B2F934CACB830DCAB6001F3CA, for 1 inputs
2016-01-14 17:16:33,108 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [ECCCCDF20D0A466A99C7A41645DB23AE] #20 in 1ms
2016-01-14 17:16:33,110 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [ECCCCDF20D0A466A99C7A41645DB23AE] #20, status:false, isInterrupted:false
2016-01-14 17:16:33,111 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:33,111 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:33,111 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:33,112 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:33,112 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:33,112 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : ECCCCDF20D0A466A99C7A41645DB23AE
2016-01-14 17:16:33,112 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FA939928E4314454B93A512D2686548D, all 1 inputs ready in: 00:00:00.004
2016-01-14 17:16:33,113 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:33,113 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@569ec4a6; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@51809ab7
2016-01-14 17:16:33,117 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:33,118 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:33,118 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791792805_0001_1_01_000000_0
2016-01-14 17:16:33,118 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791792805_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:33,118 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791792805_0001_1_01_000000_0
2016-01-14 17:16:33,118 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FA939928E4314454B93A512D2686548D, mem on close (mb), free: 382, total: 727, max: 727
2016-01-14 17:16:33,119 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: ECCCCDF20D0A466A99C7A41645DB23AE
2016-01-14 17:16:33,119 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:33,119 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:33,120 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:33,120 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:33,120 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:33 Completed running task attempt: attempt_1452791792805_0001_1_01_000000_0
2016-01-14 17:16:33,121 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791792805_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:33,122 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791792805_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FA939928E4314454B93A512D2686548D, taskAttemptId=attempt_1452791792805_0001_1_01_000000_0, startTime=1452791793075, finishTime=1452791793122, timeTaken=47, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=652, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=762314752, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=13, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791793113, Process_Duration=5, Process_End_Time=1452791793118, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:33,122 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791792805_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:33,123 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791792805_0001_1_01_000000_0
2016-01-14 17:16:33,123 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:33,123 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791792805_0001_1][Event:TASK_FINISHED]: vertexName=FA939928E4314454B93A512D2686548D, taskId=task_1452791792805_0001_1_01_000000, startTime=1452791793075, finishTime=1452791793123, timeTaken=48, status=SUCCEEDED, successfulAttemptID=attempt_1452791792805_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=652, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=762314752, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=13, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791793113, Process_Duration=5, Process_End_Time=1452791793118, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:33,123 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:33,123 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791792805_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:33,124 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791792805_0001_1_01 [FA939928E4314454B93A512D2686548D] : 1
2016-01-14 17:16:33,124 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791792805_0001_1_01 [FA939928E4314454B93A512D2686548D]
2016-01-14 17:16:33,124 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:33,124 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:33,125 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791792805_0001_1][Event:VERTEX_FINISHED]: vertexName=FA939928E4314454B93A512D2686548D, vertexId=vertex_1452791792805_0001_1_01, initRequestedTime=1452791792976, initedTime=1452791792984, startRequestedTime=1452791792989, startedTime=1452791792989, finishTime=1452791793124, timeTaken=135, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=652, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=762314752, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=13, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791793113, Process_Duration=5, Process_End_Time=1452791793118, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452791793075, firstTasksToStart=[ task_1452791792805_0001_1_01_000000 ], lastTaskFinishTime=1452791793122, lastTasksToFinish=[ task_1452791792805_0001_1_01_000000 ], minTaskDuration=47, maxTaskDuration=47, avgTaskDuration=47.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791792805_0001_1_01_000000 ], longestDurationTasks=[ task_1452791792805_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:33,125 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791792805_0001_1_01 [FA939928E4314454B93A512D2686548D] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:33,125 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791792805_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:33,125 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791792805_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791792805_0001_00_000002, stoppedTime=1452791793125, exitStatus=0
2016-01-14 17:16:33,125 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:33,126 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791792805_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:33,126 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791792805_0001_1_01 [FA939928E4314454B93A512D2686548D] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:33,126 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791792805_0001_1
2016-01-14 17:16:33,126 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791792805_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452791792805_0001_1
2016-01-14 17:16:33,126 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452791792805_0001_1_00 [ECCCCDF20D0A466A99C7A41645DB23AE]
2016-01-14 17:16:33,126 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: E7491985CB124B128CFECEDD4DFBD50B for vertex: vertex_1452791792805_0001_1_01 [FA939928E4314454B93A512D2686548D]
2016-01-14 17:16:33,128 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=652, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=762314752, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=13, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791793113, Process_Duration=5, Process_End_Time=1452791793118, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 17:16:33,128 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:33,128 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:33,128 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791792805_0001_00_000002
2016-01-14 17:16:33,128 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791792805_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 17:16:33,129 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791792805_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:33,129 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791792805_0001_00_000002, Exiting
2016-01-14 17:16:33,130 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791792805_0001_00_000002
2016-01-14 17:16:33,130 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791792805_0001_00_000002
2016-01-14 17:16:33,130 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791792805_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:33,130 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791792805_0001_00_000002 completed successfully
2016-01-14 17:16:33,136 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791792805_0001_1, queueSize=0
2016-01-14 17:16:33,137 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:33,137 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791792805_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791792805_0001_1, startTime=1452791792972, finishTime=1452791793135, timeTaken=163, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=985, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=12, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1524629504, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=13, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905583586166, Process_Duration=8, Process_End_Time=2905583586174, Read_Duration=2, Tuples_Read=40, Tuples_Written=40, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 17:16:33,137 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791792805_0001_1 finished with state: SUCCEEDED
2016-01-14 17:16:33,137 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791792805_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:33,138 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791792805_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:16:33,138 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452791792805_0001_00_000002
2016-01-14 17:16:33 Completed Dag: dag_1452791792805_0001_1
2016-01-14 17:16:33,140 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791792805_0001_1, dagState=SUCCEEDED
2016-01-14 17:16:33,140 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791792805_0001_1_post
2016-01-14 17:16:33,141 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:33,141 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:33,144 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...mblyplatform/0_'field'_-1, applicationId=application_1452791792805_0001
2016-01-14 17:16:33,146 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:33,146 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:33,146 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:33,146 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:33,146 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:33,147 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:33,147 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:33,147 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:33,149 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:33,149 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:33,149 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:33,150 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:33,150 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:33,154 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/83CDB518F31D4E4E90AA92A69D1C73B0/C8AC0F1F0F8C4AE98D272658EEFEA3BE/.tez/application_1452791792805_0001
2016-01-14 17:16:33,155 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.367
2016-01-14 17:16:33,234 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_'field'_-1
2016-01-14 17:16:33,238 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 17:16:33,238 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 17:16:33,265 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:33,265 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:33,306 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-01-14 17:16:33,313 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.022
2016-01-14 17:16:33,313 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:33,314 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:33,314 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:33,328 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:33,334 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_'field'
2016-01-14 17:16:33,377 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:33,380 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:33,380 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_'field'"]
2016-01-14 17:16:33,381 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:33,381 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:33,382 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:33,383 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...latform/0_'field'_'field'
2016-01-14 17:16:33,383 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:33,397 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:33,402 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:33,402 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:33,402 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:33,410 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/83686C13F7DA4C55B6EF73EC689058B7/D5B6C28B983D44938A7347A270EBA335/.tez/application_1452791793402_0001 doesn't exist and is created
2016-01-14 17:16:33,426 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:33,426 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:33,426 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/83686C13F7DA4C55B6EF73EC689058B7/D5B6C28B983D44938A7347A270EBA335/.tez/application_1452791793402_0001
2016-01-14 17:16:33,430 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791793402_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:33,441 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:33,441 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:33,443 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:33,443 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:33,444 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/83686C13F7DA4C55B6EF73EC689058B7/D5B6C28B983D44938A7347A270EBA335/.tez/application_1452791793402_0001/localmode-log-dir
2016-01-14 17:16:33,444 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/83686C13F7DA4C55B6EF73EC689058B7/D5B6C28B983D44938A7347A270EBA335/.tez/application_1452791793402_0001/localmode-log-dir/history.txt.appattempt_1452791793402_0001_000000, maxErrors=10
2016-01-14 17:16:33,445 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:33,445 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791793402_0001_000000, appSubmitTime=1452791793429, launchTime=1452791793430
2016-01-14 17:16:33,452 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:44555
2016-01-14 17:16:33,452 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:33,454 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:33,454 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:33,454 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:33,455 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:33,457 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791793402_0001_000000, startTime=1452791793457
2016-01-14 17:16:33,457 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:33,457 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:33,526 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:33,526 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:33,526 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/0_'field'_'field', applicationId=application_1452791793402_0001, dagName=[83686C13F7DA4C55B6EF73EC689058B7/D5B6C28B983D44938A7347A270EBA335] every/(1/1) ...latform/0_'field'_'field'
2016-01-14 17:16:33,541 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [83686C13F7DA4C55B6EF73EC689058B7/D5B6C28B983D44938A7347A270EBA335] every/(1/1) ...latform/0_'field'_'field'
2016-01-14 17:16:33,542 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791793402_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/83686C13F7DA4C55B6EF73EC689058B7/D5B6C28B983D44938A7347A270EBA335/.tez/application_1452791793402_0001/localmode-log-dir/dag_1452791793402_0001_1.dot
2016-01-14 17:16:33,543 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/83686C13F7DA4C55B6EF73EC689058B7/D5B6C28B983D44938A7347A270EBA335/.tez/application_1452791793402_0001/localmode-log-dir/dag_1452791793402_0001_1-tez-dag.pb.txt
2016-01-14 17:16:33,548 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791793402_0001_1
2016-01-14 17:16:33,548 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:33,549 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [83686C13F7DA4C55B6EF73EC689058B7/D5B6C28B983D44938A7347A270EBA335] every/(1/1) ...latform/0_'field'_'field'
2016-01-14 17:16:33 Running Dag: dag_1452791793402_0001_1
2016-01-14 17:16:33,563 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:33,563 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791793402_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791793402_0001_1, submitTime=1452791793541
2016-01-14 17:16:33,564 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex E191F0B757F2498CA9C28B99D42FEE7C
2016-01-14 17:16:33,566 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:33,566 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:33,566 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791793402_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791793402_0001_1, initTime=1452791793563
2016-01-14 17:16:33,566 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791793402_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:33,569 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791793402_0001_1][Event:DAG_STARTED]: dagID=dag_1452791793402_0001_1, startTime=1452791793569
2016-01-14 17:16:33,569 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:33,569 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:33,569 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791793402_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:33,569 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: B309157D8EDF4DEDA3F02A93642CF208 : {B84D7A7CBFB147A69982A14858EF2889={InputName=B84D7A7CBFB147A69982A14858EF2889}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:33,569 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: B84D7A7CBFB147A69982A14858EF2889, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:33,569 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/0_'field'_'field', applicationId=application_1452791793402_0001, dagName=[83686C13F7DA4C55B6EF73EC689058B7/D5B6C28B983D44938A7347A270EBA335] every/(1/1) ...latform/0_'field'_'field'
2016-01-14 17:16:33,570 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791793402_0001_1_00 [B309157D8EDF4DEDA3F02A93642CF208]
2016-01-14 17:16:33,570 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791793402_0001_1_00 [B309157D8EDF4DEDA3F02A93642CF208]
2016-01-14 17:16:33,570 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791793402_0001_1_00 [B309157D8EDF4DEDA3F02A93642CF208]
2016-01-14 17:16:33,570 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791793402_0001_1_00 [B309157D8EDF4DEDA3F02A93642CF208]. Starting root input initializers: 1
2016-01-14 17:16:33,571 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791793402_0001_1_00 [B309157D8EDF4DEDA3F02A93642CF208] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:33,571 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791793402_0001_1_01 [E191F0B757F2498CA9C28B99D42FEE7C]
2016-01-14 17:16:33,573 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: B84D7A7CBFB147A69982A14858EF2889 on vertex vertex_1452791793402_0001_1_00 [B309157D8EDF4DEDA3F02A93642CF208]
2016-01-14 17:16:33,574 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:33,574 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791793402_0001_1_01 [E191F0B757F2498CA9C28B99D42FEE7C]
2016-01-14 17:16:33,574 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791793402_0001_1_01 [E191F0B757F2498CA9C28B99D42FEE7C]
2016-01-14 17:16:33,574 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791793402_0001_1_01 [E191F0B757F2498CA9C28B99D42FEE7C]
2016-01-14 17:16:33,574 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=31D49C6A954B48C287CC4DB92A9A880F, vertexId=vertex_1452791793402_0001_1_01 [E191F0B757F2498CA9C28B99D42FEE7C], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:33,574 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=31D49C6A954B48C287CC4DB92A9A880F, vertexId=vertex_1452791793402_0001_1_01 [E191F0B757F2498CA9C28B99D42FEE7C]
2016-01-14 17:16:33,576 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=31D49C6A954B48C287CC4DB92A9A880F, vertexName=E191F0B757F2498CA9C28B99D42FEE7C, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:33,576 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=31D49C6A954B48C287CC4DB92A9A880F, vertexId=vertex_1452791793402_0001_1_01 [E191F0B757F2498CA9C28B99D42FEE7C]
2016-01-14 17:16:33,578 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:33,578 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791793402_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E191F0B757F2498CA9C28B99D42FEE7C, vertexId=vertex_1452791793402_0001_1_01, initRequestedTime=1452791793571, initedTime=1452791793578, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:33,578 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791793402_0001_1_01 [E191F0B757F2498CA9C28B99D42FEE7C] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:33,574 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:33,579 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input B84D7A7CBFB147A69982A14858EF2889 asking for 1 tasks. Headroom: 727 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:33,579 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:33,580 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:33,581 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:33,581 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: B84D7A7CBFB147A69982A14858EF2889 on vertex vertex_1452791793402_0001_1_00 [B309157D8EDF4DEDA3F02A93642CF208]
2016-01-14 17:16:33,582 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791793402_0001_1_00 [B309157D8EDF4DEDA3F02A93642CF208] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:33,582 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {B84D7A7CBFB147A69982A14858EF2889=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:33,582 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791793402_0001_1_00 [B309157D8EDF4DEDA3F02A93642CF208] parallelism set to 1
2016-01-14 17:16:33,583 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791793402_0001_1_00 [B309157D8EDF4DEDA3F02A93642CF208] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:33,583 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791793402_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791793402_0001_1_00, eventCount=1
2016-01-14 17:16:33,583 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:33,584 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791793402_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B309157D8EDF4DEDA3F02A93642CF208, vertexId=vertex_1452791793402_0001_1_00, initRequestedTime=1452791793569, initedTime=1452791793583, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:33,584 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791793402_0001_1_00 [B309157D8EDF4DEDA3F02A93642CF208] with distanceFromRoot: 0
2016-01-14 17:16:33,584 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791793402_0001_1_00 [B309157D8EDF4DEDA3F02A93642CF208] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:33,584 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in B309157D8EDF4DEDA3F02A93642CF208
2016-01-14 17:16:33,585 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791793402_0001_1_00 [B309157D8EDF4DEDA3F02A93642CF208]
2016-01-14 17:16:33,585 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:33,585 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791793402_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791793402_0001_1_00, startRequestedTime=1452791793584, startedTime=1452791793584
2016-01-14 17:16:33,586 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791793402_0001_1_00 [B309157D8EDF4DEDA3F02A93642CF208] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:33,586 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:33,586 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791793402_0001_1][Event:TASK_STARTED]: vertexName=B309157D8EDF4DEDA3F02A93642CF208, taskId=task_1452791793402_0001_1_00_000000, scheduledTime=1452791793586, launchTime=1452791793586
2016-01-14 17:16:33,587 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791793402_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:33,587 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791793402_0001_1_00 for vertex: vertex_1452791793402_0001_1_01 [E191F0B757F2498CA9C28B99D42FEE7C] numStartedSources: 1 numSources: 1
2016-01-14 17:16:33,587 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791793402_0001_1_01 [E191F0B757F2498CA9C28B99D42FEE7C] with distanceFromRoot: 1
2016-01-14 17:16:33,588 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791793402_0001_1_00 [B309157D8EDF4DEDA3F02A93642CF208]
2016-01-14 17:16:33,588 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: B309157D8EDF4DEDA3F02A93642CF208 in vertex: E191F0B757F2498CA9C28B99D42FEE7C
2016-01-14 17:16:33,588 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: E191F0B757F2498CA9C28B99D42FEE7C with 1 source tasks and 1 pending tasks
2016-01-14 17:16:33,589 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:33,589 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791793402_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791793402_0001_1_01, startRequestedTime=1452791793588, startedTime=1452791793588
2016-01-14 17:16:33,589 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791793402_0001_1_01 [E191F0B757F2498CA9C28B99D42FEE7C] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:33,590 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [83686C13F7DA4C55B6EF73EC689058B7/D5B6C28B983D44938A7347A270EBA335] every/(1/1) ...latform/0_'field'_'field', VertexName: B309157D8EDF4DEDA3F02A93642CF208, VertexParallelism: 1, TaskAttemptID:attempt_1452791793402_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B84D7A7CBFB147A69982A14858EF2889, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E191F0B757F2498CA9C28B99D42FEE7C, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:33,590 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791793402_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:33,590 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:33,588 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791793402_0001, with dag id: dag_1452791793402_0001_1
2016-01-14 17:16:33,594 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:33,594 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:33,594 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:33,595 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791793402_0001_00_000001
2016-01-14 17:16:33,595 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791793402_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:33,596 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791793402_0001_00_000001
2016-01-14 17:16:33,597 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:33,597 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791793402_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:33,597 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791793402_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791793402_0001_00_000001, launchTime=1452791793597
2016-01-14 17:16:33,597 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:33,598 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791793402_0001_00_000001
2016-01-14 17:16:33,599 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791793402_0001_1_00_000000_0] to container: [container_1452791793402_0001_00_000001]
2016-01-14 17:16:33,599 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791793402_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:33,599 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791793402_0001_00_000001 given task: attempt_1452791793402_0001_1_00_000000_0
2016-01-14 17:16:33,599 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791793402_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791793402_0001_1_00_000000_0
2016-01-14 17:16:33,599 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791793402_0001_1_00_000000_0] started. Is using containerId: [container_1452791793402_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:33 Starting to run new task attempt: attempt_1452791793402_0001_1_00_000000_0
2016-01-14 17:16:33,600 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791793402_0001_1_00_000000_0
2016-01-14 17:16:33,600 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791793402_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B309157D8EDF4DEDA3F02A93642CF208, taskAttemptId=attempt_1452791793402_0001_1_00_000000_0, startTime=1452791793599, containerId=container_1452791793402_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791793402_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:33,600 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:33,601 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791793402_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:33,601 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791793402_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:33,601 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:33,602 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:33,602 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:33,602 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:33,602 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:33,602 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:33,602 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [83686C13F7DA4C55B6EF73EC689058B7/D5B6C28B983D44938A7347A270EBA335] every/(1/1) ...latform/0_'field'_'field', VertexName: B309157D8EDF4DEDA3F02A93642CF208, VertexParallelism: 1, TaskAttemptID:attempt_1452791793402_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B84D7A7CBFB147A69982A14858EF2889, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E191F0B757F2498CA9C28B99D42FEE7C, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:33,603 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=762314752
2016-01-14 17:16:33,603 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791793402_0001_1_00_000000_0
2016-01-14 17:16:33,603 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:33,603 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791793402_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:33,603 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:33,604 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:33,604 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B84D7A7CBFB147A69982A14858EF2889, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:33,604 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=E191F0B757F2498CA9C28B99D42FEE7C, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:33,605 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:33,605 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B84D7A7CBFB147A69982A14858EF2889
2016-01-14 17:16:33,605 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: B84D7A7CBFB147A69982A14858EF2889
2016-01-14 17:16:33,605 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:33,606 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: E191F0B757F2498CA9C28B99D42FEE7C
2016-01-14 17:16:33,607 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:33,607 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: B84D7A7CBFB147A69982A14858EF2889
2016-01-14 17:16:33,607 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B84D7A7CBFB147A69982A14858EF2889
2016-01-14 17:16:33,607 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:33,607 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: E191F0B757F2498CA9C28B99D42FEE7C
2016-01-14 17:16:33,608 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: E191F0B757F2498CA9C28B99D42FEE7C
2016-01-14 17:16:33,609 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:33,614 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B309157D8EDF4DEDA3F02A93642CF208, ordinal: 0
2016-01-14 17:16:33,614 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B309157D8EDF4DEDA3F02A93642CF208, mem on start (mb), free: 288, total: 727, max: 727
2016-01-14 17:16:33,614 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:33,614 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:33,614 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:33,615 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:33,615 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:33,615 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:33,615 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:33,615 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:33,616 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:33,616 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 762314752, TotalAvailable: 510750883, TotalRequested/TotalJVMHeap:0.14
2016-01-14 17:16:33,616 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B84D7A7CBFB147A69982A14858EF2889, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:33,616 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, E191F0B757F2498CA9C28B99D42FEE7C, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:33,616 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:33,616 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B84D7A7CBFB147A69982A14858EF2889 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:33,617 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:33,617 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:33,616 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B84D7A7CBFB147A69982A14858EF2889
2016-01-14 17:16:33,617 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:33,617 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B84D7A7CBFB147A69982A14858EF2889
2016-01-14 17:16:33,618 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:33,618 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791793402_0001_1_00_000000_0
2016-01-14 17:16:33,618 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:33,619 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@114f03b
2016-01-14 17:16:33,620 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@114f03b
2016-01-14 17:16:33,620 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:33,620 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:33,621 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:33,622 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: B84D7A7CBFB147A69982A14858EF2889
2016-01-14 17:16:33,622 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: C9A463076B534F8AA186A6FD0DCED531
2016-01-14 17:16:33,623 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] C9A463076B534F8AA186A6FD0DCED531
2016-01-14 17:16:33,624 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@57657040; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@44aca693; comparator=cascading.tuple.hadoop.util.TupleComparator@22ec201e; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:33,624 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:33,624 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:33,624 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:33,648 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:33,648 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:33,649 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:33,649 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:33,649 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:33,650 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:33,650 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:33,650 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B309157D8EDF4DEDA3F02A93642CF208, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:33,653 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791793402_0001_1_00_000000_0
2016-01-14 17:16:33,653 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B309157D8EDF4DEDA3F02A93642CF208, mem on close (mb), free: 388, total: 726, max: 726
2016-01-14 17:16:33,653 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:33,654 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:33,654 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:33,654 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:33,657 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:33 Completed running task attempt: attempt_1452791793402_0001_1_00_000000_0
2016-01-14 17:16:33,660 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791793402_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:33,660 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791793402_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791793402_0001_1_00, eventCount=1
2016-01-14 17:16:33,660 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:33,660 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791793402_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B309157D8EDF4DEDA3F02A93642CF208, taskAttemptId=attempt_1452791793402_0001_1_00_000000_0, startTime=1452791793599, finishTime=1452791793660, timeTaken=61, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=761790464, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791793650, Process_Duration=3, Process_End_Time=1452791793653, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:33,661 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791793402_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:33,661 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791793402_0001_1_00_000000_0
2016-01-14 17:16:33,661 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:33,662 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791793402_0001_1][Event:TASK_FINISHED]: vertexName=B309157D8EDF4DEDA3F02A93642CF208, taskId=task_1452791793402_0001_1_00_000000, startTime=1452791793599, finishTime=1452791793661, timeTaken=62, status=SUCCEEDED, successfulAttemptID=attempt_1452791793402_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=761790464, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791793650, Process_Duration=3, Process_End_Time=1452791793653, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:33,662 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:33,662 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791793402_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:33,662 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791793402_0001_1_00 [B309157D8EDF4DEDA3F02A93642CF208] : 1
2016-01-14 17:16:33,662 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791793402_0001_1_00 [B309157D8EDF4DEDA3F02A93642CF208]
2016-01-14 17:16:33,663 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:33,663 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791793402_0001_1][Event:VERTEX_FINISHED]: vertexName=B309157D8EDF4DEDA3F02A93642CF208, vertexId=vertex_1452791793402_0001_1_00, initRequestedTime=1452791793569, initedTime=1452791793583, startRequestedTime=1452791793584, startedTime=1452791793584, finishTime=1452791793662, timeTaken=78, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=761790464, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791793650, Process_Duration=3, Process_End_Time=1452791793653, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791793599, firstTasksToStart=[ task_1452791793402_0001_1_00_000000 ], lastTaskFinishTime=1452791793660, lastTasksToFinish=[ task_1452791793402_0001_1_00_000000 ], minTaskDuration=61, maxTaskDuration=61, avgTaskDuration=61.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791793402_0001_1_00_000000 ], longestDurationTasks=[ task_1452791793402_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:33,663 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791793402_0001_1_00 [B309157D8EDF4DEDA3F02A93642CF208] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:33,664 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791793402_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:33,664 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791793402_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791793402_0001_00_000001, stoppedTime=1452791793664, exitStatus=0
2016-01-14 17:16:33,664 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791793402_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:33,664 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791793402_0001_1_01 [E191F0B757F2498CA9C28B99D42FEE7C] attempt: attempt_1452791793402_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:33,664 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: E191F0B757F2498CA9C28B99D42FEE7C
2016-01-14 17:16:33,665 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791793402_0001_1_01 [E191F0B757F2498CA9C28B99D42FEE7C]
2016-01-14 17:16:33,665 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791793402_0001_1_00 [B309157D8EDF4DEDA3F02A93642CF208] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:33,665 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791793402_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:33,665 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791793402_0001_1][Event:TASK_STARTED]: vertexName=E191F0B757F2498CA9C28B99D42FEE7C, taskId=task_1452791793402_0001_1_01_000000, scheduledTime=1452791793665, launchTime=1452791793665
2016-01-14 17:16:33,665 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791793402_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:33,666 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791793402_0001_1_01 [E191F0B757F2498CA9C28B99D42FEE7C]
2016-01-14 17:16:33,666 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791793402_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:33,666 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [83686C13F7DA4C55B6EF73EC689058B7/D5B6C28B983D44938A7347A270EBA335] every/(1/1) ...latform/0_'field'_'field', VertexName: E191F0B757F2498CA9C28B99D42FEE7C, VertexParallelism: 1, TaskAttemptID:attempt_1452791793402_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B309157D8EDF4DEDA3F02A93642CF208, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=31D49C6A954B48C287CC4DB92A9A880F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:33,666 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791793402_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:33,666 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:33,666 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:33,666 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:33,667 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=761790464, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791793650, Process_Duration=3, Process_End_Time=1452791793653, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:33,667 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791793402_0001_00_000002
2016-01-14 17:16:33,667 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:33,667 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:33,667 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791793402_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:33,668 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791793402_0001_00_000001
2016-01-14 17:16:33,668 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791793402_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:33,668 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791793402_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:33,668 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791793402_0001_00_000001, Exiting
2016-01-14 17:16:33,668 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791793402_0001_00_000001
2016-01-14 17:16:33,669 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791793402_0001_00_000001
2016-01-14 17:16:33,669 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791793402_0001_00_000002
2016-01-14 17:16:33,670 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791793402_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:33,670 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791793402_0001_00_000001 completed successfully
2016-01-14 17:16:33,670 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791793402_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:33,670 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 17:16:33,672 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791793402_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:33,672 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:33,673 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791793402_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791793402_0001_00_000002, launchTime=1452791793672
2016-01-14 17:16:33,673 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791793402_0001_00_000002
2016-01-14 17:16:33,674 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791793402_0001_1_01_000000_0] to container: [container_1452791793402_0001_00_000002]
2016-01-14 17:16:33,674 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791793402_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:33,674 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791793402_0001_00_000002 given task: attempt_1452791793402_0001_1_01_000000_0
2016-01-14 17:16:33,674 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791793402_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791793402_0001_1_01_000000_0
2016-01-14 17:16:33,674 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791793402_0001_1_01_000000_0] started. Is using containerId: [container_1452791793402_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:33 Starting to run new task attempt: attempt_1452791793402_0001_1_01_000000_0
2016-01-14 17:16:33,674 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791793402_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E191F0B757F2498CA9C28B99D42FEE7C, taskAttemptId=attempt_1452791793402_0001_1_01_000000_0, startTime=1452791793674, containerId=container_1452791793402_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791793402_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:33,674 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791793402_0001_1_01_000000_0
2016-01-14 17:16:33,674 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791793402_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:33,675 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791793402_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:33,675 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:33,675 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:33,675 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:33,675 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:33,675 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:33,676 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:33,676 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:33,676 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [83686C13F7DA4C55B6EF73EC689058B7/D5B6C28B983D44938A7347A270EBA335] every/(1/1) ...latform/0_'field'_'field', VertexName: E191F0B757F2498CA9C28B99D42FEE7C, VertexParallelism: 1, TaskAttemptID:attempt_1452791793402_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B309157D8EDF4DEDA3F02A93642CF208, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=31D49C6A954B48C287CC4DB92A9A880F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:33,676 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=761790464
2016-01-14 17:16:33,677 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791793402_0001_1_01_000000_0
2016-01-14 17:16:33,677 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:33,677 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:33,677 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791793402_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:33,678 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:33,678 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=31D49C6A954B48C287CC4DB92A9A880F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:33,679 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:33,679 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 31D49C6A954B48C287CC4DB92A9A880F
2016-01-14 17:16:33,679 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:33,679 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 31D49C6A954B48C287CC4DB92A9A880F
2016-01-14 17:16:33,681 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:33,686 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B309157D8EDF4DEDA3F02A93642CF208, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:33,687 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:33,687 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B309157D8EDF4DEDA3F02A93642CF208
2016-01-14 17:16:33,688 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E191F0B757F2498CA9C28B99D42FEE7C, ordinal: 1
2016-01-14 17:16:33,688 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E191F0B757F2498CA9C28B99D42FEE7C, mem on start (mb), free: 383, total: 726, max: 726
2016-01-14 17:16:33,689 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:33,689 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:33,689 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 685611392, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:33,690 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:33,690 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 685611392, type: INPUT, componentVertexName: B309157D8EDF4DEDA3F02A93642CF208
2016-01-14 17:16:33,690 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B309157D8EDF4DEDA3F02A93642CF208
2016-01-14 17:16:33,690 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:33,691 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:33,691 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 31D49C6A954B48C287CC4DB92A9A880F
2016-01-14 17:16:33,691 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:33,691 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:33,691 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:33,692 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:33,692 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:33,692 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:33,692 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 685611392, TotalRequestedScaled: 6.328720541538461E8, TotalJVMHeap: 761790464, TotalAvailable: 510399610, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:33,692 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 31D49C6A954B48C287CC4DB92A9A880F, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:33,692 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B309157D8EDF4DEDA3F02A93642CF208, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=685611392, allocated=510399610
2016-01-14 17:16:33,692 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:33,692 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B309157D8EDF4DEDA3F02A93642CF208 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:33,693 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:33,693 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:33,692 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B309157D8EDF4DEDA3F02A93642CF208
2016-01-14 17:16:33,694 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:33,694 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:33,694 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: B309157D8EDF4DEDA3F02A93642CF208 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:33,694 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=685611392, postMergeMem=0, RuntimeTotalAvailable=510399610. Updated to: ShuffleMem=510399610, postMergeMem: 0
2016-01-14 17:16:33,695 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=510399610, maxSingleShuffleLimit=127599904, mergeThreshold=459359648, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:33,695 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:33,697 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B309157D8EDF4DEDA3F02A93642CF208
2016-01-14 17:16:33,697 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:33,698 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791793402_0001_1_01_000000_0
2016-01-14 17:16:33,700 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:33,701 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: C9A463076B534F8AA186A6FD0DCED531
2016-01-14 17:16:33,701 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_'field'"], id: 31D49C6A954B48C287CC4DB92A9A880F
2016-01-14 17:16:33,701 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791793402_0001_1_00_000000_0_10123, runDuration: 52673, ]
2016-01-14 17:16:33,701 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_'field'"]
2016-01-14 17:16:33,701 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] C9A463076B534F8AA186A6FD0DCED531, for 1 inputs
2016-01-14 17:16:33,702 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791793402_0001_1_00_000000_0_10123] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791793702, TimeTaken=0, Rate=0.00 MB/s
2016-01-14 17:16:33,702 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : B309157D8EDF4DEDA3F02A93642CF208
2016-01-14 17:16:33,702 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:33,703 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [B309157D8EDF4DEDA3F02A93642CF208] #21 in 2ms
2016-01-14 17:16:33,703 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [B309157D8EDF4DEDA3F02A93642CF208] #21, status:false, isInterrupted:false
2016-01-14 17:16:33,703 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:33,703 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:33,703 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:33,703 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:33,704 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:33,704 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : B309157D8EDF4DEDA3F02A93642CF208
2016-01-14 17:16:33,704 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:33,704 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E191F0B757F2498CA9C28B99D42FEE7C, all 1 inputs ready in: 00:00:00.003
2016-01-14 17:16:33,704 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@680bcc94; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@5ea43de9
2016-01-14 17:16:33,707 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:33,707 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:33,708 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791793402_0001_1_01_000000_0
2016-01-14 17:16:33,708 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791793402_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:33,708 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791793402_0001_1_01_000000_0
2016-01-14 17:16:33,708 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E191F0B757F2498CA9C28B99D42FEE7C, mem on close (mb), free: 380, total: 726, max: 726
2016-01-14 17:16:33,709 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: B309157D8EDF4DEDA3F02A93642CF208
2016-01-14 17:16:33,709 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:33,709 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:33,709 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:33,709 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:33,710 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:33 Completed running task attempt: attempt_1452791793402_0001_1_01_000000_0
2016-01-14 17:16:33,710 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791793402_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:33,711 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=761790464, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791793704, Process_Duration=4, Process_End_Time=1452791793708, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 17:16:33,711 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:33,711 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:33,711 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:33,711 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791793402_0001_00_000002
2016-01-14 17:16:33,711 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791793402_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E191F0B757F2498CA9C28B99D42FEE7C, taskAttemptId=attempt_1452791793402_0001_1_01_000000_0, startTime=1452791793674, finishTime=1452791793711, timeTaken=37, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=761790464, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791793704, Process_Duration=4, Process_End_Time=1452791793708, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:33,712 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791793402_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:33,712 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:33,712 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791793402_0001_1_01_000000_0
2016-01-14 17:16:33,713 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:33,713 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791793402_0001_1][Event:TASK_FINISHED]: vertexName=E191F0B757F2498CA9C28B99D42FEE7C, taskId=task_1452791793402_0001_1_01_000000, startTime=1452791793674, finishTime=1452791793712, timeTaken=38, status=SUCCEEDED, successfulAttemptID=attempt_1452791793402_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=761790464, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791793704, Process_Duration=4, Process_End_Time=1452791793708, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:33,713 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791793402_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:33,713 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791793402_0001_1_01 [E191F0B757F2498CA9C28B99D42FEE7C] : 1
2016-01-14 17:16:33,713 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791793402_0001_1_01 [E191F0B757F2498CA9C28B99D42FEE7C]
2016-01-14 17:16:33,714 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:33,714 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791793402_0001_1][Event:VERTEX_FINISHED]: vertexName=E191F0B757F2498CA9C28B99D42FEE7C, vertexId=vertex_1452791793402_0001_1_01, initRequestedTime=1452791793571, initedTime=1452791793578, startRequestedTime=1452791793588, startedTime=1452791793588, finishTime=1452791793713, timeTaken=125, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=761790464, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791793704, Process_Duration=4, Process_End_Time=1452791793708, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452791793674, firstTasksToStart=[ task_1452791793402_0001_1_01_000000 ], lastTaskFinishTime=1452791793711, lastTasksToFinish=[ task_1452791793402_0001_1_01_000000 ], minTaskDuration=37, maxTaskDuration=37, avgTaskDuration=37.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791793402_0001_1_01_000000 ], longestDurationTasks=[ task_1452791793402_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:33,715 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791793402_0001_1_01 [E191F0B757F2498CA9C28B99D42FEE7C] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:33,715 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791793402_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:33,715 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791793402_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791793402_0001_00_000002, stoppedTime=1452791793715, exitStatus=0
2016-01-14 17:16:33,715 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:33,715 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791793402_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:33,715 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791793402_0001_1_01 [E191F0B757F2498CA9C28B99D42FEE7C] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:33,716 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791793402_0001_1
2016-01-14 17:16:33,716 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791793402_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452791793402_0001_1
2016-01-14 17:16:33,716 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452791793402_0001_1_00 [B309157D8EDF4DEDA3F02A93642CF208]
2016-01-14 17:16:33,716 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 31D49C6A954B48C287CC4DB92A9A880F for vertex: vertex_1452791793402_0001_1_01 [E191F0B757F2498CA9C28B99D42FEE7C]
2016-01-14 17:16:33,722 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791793402_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 17:16:33,722 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791793402_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:33,722 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791793402_0001_00_000002, Exiting
2016-01-14 17:16:33,722 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791793402_0001_00_000002
2016-01-14 17:16:33,722 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791793402_0001_00_000002
2016-01-14 17:16:33,723 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791793402_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:33,723 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791793402_0001_00_000002 completed successfully
2016-01-14 17:16:33,727 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791793402_0001_1, queueSize=0
2016-01-14 17:16:33,727 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:33,728 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791793402_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791793402_0001_1, startTime=1452791793569, finishTime=1452791793725, timeTaken=156, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=465, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1523580928, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905583587354, Process_Duration=7, Process_End_Time=2905583587361, Read_Duration=1, Tuples_Read=40, Tuples_Written=40, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 17:16:33,728 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791793402_0001_1 finished with state: SUCCEEDED
2016-01-14 17:16:33,728 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791793402_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:33,728 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452791793402_0001_00_000002
2016-01-14 17:16:33,728 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791793402_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:16:33 Completed Dag: dag_1452791793402_0001_1
2016-01-14 17:16:33,729 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791793402_0001_1, dagState=SUCCEEDED
2016-01-14 17:16:33,730 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791793402_0001_1_post
2016-01-14 17:16:33,730 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:33,730 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:33,737 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/0_'field'_'field', applicationId=application_1452791793402_0001
2016-01-14 17:16:33,739 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:33,739 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:33,739 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:33,739 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:33,739 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:33,739 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:33,739 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:33,740 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:33,740 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:33,740 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:33,740 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:33,741 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:33,741 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:33,749 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/83686C13F7DA4C55B6EF73EC689058B7/D5B6C28B983D44938A7347A270EBA335/.tez/application_1452791793402_0001
2016-01-14 17:16:33,750 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.372
2016-01-14 17:16:33,829 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_'field'_'field'
2016-01-14 17:16:33,852 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:33,852 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:33,886 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.011
2016-01-14 17:16:33,889 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-01-14 17:16:33,890 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:33,890 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:33,890 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:33,903 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:33,908 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_RESULTS
2016-01-14 17:16:33,951 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:33,951 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:33,951 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_RESULTS"]
2016-01-14 17:16:33,952 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:33,954 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:33,954 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:33,955 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...latform/0_'field'_RESULTS
2016-01-14 17:16:33,955 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:33,956 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 17:16:33,956 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 17:16:33,966 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:33,970 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:33,971 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:33,971 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:33,980 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/BFF19B0CF0E2479593D6FB069DA351B5/D9444EA0E17B43E7B2F2119C2BC9739E/.tez/application_1452791793971_0001 doesn't exist and is created
2016-01-14 17:16:33,996 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:33,996 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:33,997 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/BFF19B0CF0E2479593D6FB069DA351B5/D9444EA0E17B43E7B2F2119C2BC9739E/.tez/application_1452791793971_0001
2016-01-14 17:16:34,001 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791793971_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:34,011 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:34,011 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:34,013 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:34,013 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:34,014 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/BFF19B0CF0E2479593D6FB069DA351B5/D9444EA0E17B43E7B2F2119C2BC9739E/.tez/application_1452791793971_0001/localmode-log-dir
2016-01-14 17:16:34,014 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/BFF19B0CF0E2479593D6FB069DA351B5/D9444EA0E17B43E7B2F2119C2BC9739E/.tez/application_1452791793971_0001/localmode-log-dir/history.txt.appattempt_1452791793971_0001_000000, maxErrors=10
2016-01-14 17:16:34,015 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:34,015 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791793971_0001_000000, appSubmitTime=1452791794000, launchTime=1452791794000
2016-01-14 17:16:34,019 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:34,020 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:34,020 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:34,021 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:34,024 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:59121
2016-01-14 17:16:34,025 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:34,026 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791793971_0001_000000, startTime=1452791794026
2016-01-14 17:16:34,026 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:34,027 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:34,097 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:34,097 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:34,097 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/0_'field'_RESULTS, applicationId=application_1452791793971_0001, dagName=[BFF19B0CF0E2479593D6FB069DA351B5/D9444EA0E17B43E7B2F2119C2BC9739E] every/(1/1) ...latform/0_'field'_RESULTS
2016-01-14 17:16:34,112 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [BFF19B0CF0E2479593D6FB069DA351B5/D9444EA0E17B43E7B2F2119C2BC9739E] every/(1/1) ...latform/0_'field'_RESULTS
2016-01-14 17:16:34,113 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791793971_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/BFF19B0CF0E2479593D6FB069DA351B5/D9444EA0E17B43E7B2F2119C2BC9739E/.tez/application_1452791793971_0001/localmode-log-dir/dag_1452791793971_0001_1.dot
2016-01-14 17:16:34,113 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/BFF19B0CF0E2479593D6FB069DA351B5/D9444EA0E17B43E7B2F2119C2BC9739E/.tez/application_1452791793971_0001/localmode-log-dir/dag_1452791793971_0001_1-tez-dag.pb.txt
2016-01-14 17:16:34,119 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791793971_0001_1
2016-01-14 17:16:34,119 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:34,119 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [BFF19B0CF0E2479593D6FB069DA351B5/D9444EA0E17B43E7B2F2119C2BC9739E] every/(1/1) ...latform/0_'field'_RESULTS
2016-01-14 17:16:34 Running Dag: dag_1452791793971_0001_1
2016-01-14 17:16:34,130 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:34,131 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791793971_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791793971_0001_1, submitTime=1452791794112
2016-01-14 17:16:34,131 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex EFD18C74FFB34E38BB0B32FAB02F3C01
2016-01-14 17:16:34,131 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:34,131 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:34,132 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791793971_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791793971_0001_1, initTime=1452791794131
2016-01-14 17:16:34,132 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791793971_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:34,132 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791793971_0001_1][Event:DAG_STARTED]: dagID=dag_1452791793971_0001_1, startTime=1452791794132
2016-01-14 17:16:34,132 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:34,132 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:34,132 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791793971_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:34,133 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: A403C8FC1FCE4446837A056F98F9ACBF : {2BAB98386D2D4BAA86715C87116A6E33={InputName=2BAB98386D2D4BAA86715C87116A6E33}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:34,133 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 2BAB98386D2D4BAA86715C87116A6E33, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:34,133 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791793971_0001_1_00 [A403C8FC1FCE4446837A056F98F9ACBF]
2016-01-14 17:16:34,133 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791793971_0001_1_00 [A403C8FC1FCE4446837A056F98F9ACBF]
2016-01-14 17:16:34,133 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791793971_0001_1_00 [A403C8FC1FCE4446837A056F98F9ACBF]
2016-01-14 17:16:34,133 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/0_'field'_RESULTS, applicationId=application_1452791793971_0001, dagName=[BFF19B0CF0E2479593D6FB069DA351B5/D9444EA0E17B43E7B2F2119C2BC9739E] every/(1/1) ...latform/0_'field'_RESULTS
2016-01-14 17:16:34,133 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791793971_0001_1_00 [A403C8FC1FCE4446837A056F98F9ACBF]. Starting root input initializers: 1
2016-01-14 17:16:34,134 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791793971_0001_1_00 [A403C8FC1FCE4446837A056F98F9ACBF] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:34,134 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791793971_0001_1_01 [EFD18C74FFB34E38BB0B32FAB02F3C01]
2016-01-14 17:16:34,136 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 2BAB98386D2D4BAA86715C87116A6E33 on vertex vertex_1452791793971_0001_1_00 [A403C8FC1FCE4446837A056F98F9ACBF]
2016-01-14 17:16:34,137 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:34,137 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791793971_0001_1_01 [EFD18C74FFB34E38BB0B32FAB02F3C01]
2016-01-14 17:16:34,137 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791793971_0001_1_01 [EFD18C74FFB34E38BB0B32FAB02F3C01]
2016-01-14 17:16:34,137 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791793971_0001_1_01 [EFD18C74FFB34E38BB0B32FAB02F3C01]
2016-01-14 17:16:34,137 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=A00C5EEF886643E59D447868E0FBD3A0, vertexId=vertex_1452791793971_0001_1_01 [EFD18C74FFB34E38BB0B32FAB02F3C01], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:34,138 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=A00C5EEF886643E59D447868E0FBD3A0, vertexId=vertex_1452791793971_0001_1_01 [EFD18C74FFB34E38BB0B32FAB02F3C01]
2016-01-14 17:16:34,139 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:34,139 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 2BAB98386D2D4BAA86715C87116A6E33 asking for 1 tasks. Headroom: 726 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:34,139 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:34,140 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:34,140 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=A00C5EEF886643E59D447868E0FBD3A0, vertexName=EFD18C74FFB34E38BB0B32FAB02F3C01, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:34,141 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=A00C5EEF886643E59D447868E0FBD3A0, vertexId=vertex_1452791793971_0001_1_01 [EFD18C74FFB34E38BB0B32FAB02F3C01]
2016-01-14 17:16:34,142 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:34,142 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:34,142 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791793971_0001_1][Event:VERTEX_INITIALIZED]: vertexName=EFD18C74FFB34E38BB0B32FAB02F3C01, vertexId=vertex_1452791793971_0001_1_01, initRequestedTime=1452791794134, initedTime=1452791794142, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:34,142 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791793971_0001_1_01 [EFD18C74FFB34E38BB0B32FAB02F3C01] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:34,143 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 2BAB98386D2D4BAA86715C87116A6E33 on vertex vertex_1452791793971_0001_1_00 [A403C8FC1FCE4446837A056F98F9ACBF]
2016-01-14 17:16:34,143 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791793971_0001_1_00 [A403C8FC1FCE4446837A056F98F9ACBF] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:34,143 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {2BAB98386D2D4BAA86715C87116A6E33=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:34,144 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791793971_0001_1_00 [A403C8FC1FCE4446837A056F98F9ACBF] parallelism set to 1
2016-01-14 17:16:34,144 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791793971_0001_1_00 [A403C8FC1FCE4446837A056F98F9ACBF] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:34,144 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791793971_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791793971_0001_1_00, eventCount=1
2016-01-14 17:16:34,145 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:34,145 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791793971_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A403C8FC1FCE4446837A056F98F9ACBF, vertexId=vertex_1452791793971_0001_1_00, initRequestedTime=1452791794132, initedTime=1452791794145, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:34,145 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791793971_0001_1_00 [A403C8FC1FCE4446837A056F98F9ACBF] with distanceFromRoot: 0
2016-01-14 17:16:34,145 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791793971_0001_1_00 [A403C8FC1FCE4446837A056F98F9ACBF] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:34,146 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in A403C8FC1FCE4446837A056F98F9ACBF
2016-01-14 17:16:34,146 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791793971_0001_1_00 [A403C8FC1FCE4446837A056F98F9ACBF]
2016-01-14 17:16:34,146 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:34,146 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791793971_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791793971_0001_1_00, startRequestedTime=1452791794146, startedTime=1452791794146
2016-01-14 17:16:34,147 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791793971_0001_1_00 [A403C8FC1FCE4446837A056F98F9ACBF] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:34,147 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:34,147 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791793971_0001_1][Event:TASK_STARTED]: vertexName=A403C8FC1FCE4446837A056F98F9ACBF, taskId=task_1452791793971_0001_1_00_000000, scheduledTime=1452791794147, launchTime=1452791794147
2016-01-14 17:16:34,147 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791793971_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:34,148 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791793971_0001_1_00 for vertex: vertex_1452791793971_0001_1_01 [EFD18C74FFB34E38BB0B32FAB02F3C01] numStartedSources: 1 numSources: 1
2016-01-14 17:16:34,148 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791793971_0001_1_01 [EFD18C74FFB34E38BB0B32FAB02F3C01] with distanceFromRoot: 1
2016-01-14 17:16:34,148 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791793971_0001_1_00 [A403C8FC1FCE4446837A056F98F9ACBF]
2016-01-14 17:16:34,148 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: A403C8FC1FCE4446837A056F98F9ACBF in vertex: EFD18C74FFB34E38BB0B32FAB02F3C01
2016-01-14 17:16:34,149 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: EFD18C74FFB34E38BB0B32FAB02F3C01 with 1 source tasks and 1 pending tasks
2016-01-14 17:16:34,149 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:34,149 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791793971_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791793971_0001_1_01, startRequestedTime=1452791794148, startedTime=1452791794148
2016-01-14 17:16:34,149 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791793971_0001_1_01 [EFD18C74FFB34E38BB0B32FAB02F3C01] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:34,150 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [BFF19B0CF0E2479593D6FB069DA351B5/D9444EA0E17B43E7B2F2119C2BC9739E] every/(1/1) ...latform/0_'field'_RESULTS, VertexName: A403C8FC1FCE4446837A056F98F9ACBF, VertexParallelism: 1, TaskAttemptID:attempt_1452791793971_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=2BAB98386D2D4BAA86715C87116A6E33, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=EFD18C74FFB34E38BB0B32FAB02F3C01, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:34,150 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791793971_0001, with dag id: dag_1452791793971_0001_1
2016-01-14 17:16:34,150 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791793971_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:34,150 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:34,151 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:34,151 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:34,151 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:34,152 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791793971_0001_00_000001
2016-01-14 17:16:34,152 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791793971_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:34,153 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791793971_0001_00_000001
2016-01-14 17:16:34,153 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:34,154 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791793971_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:34,154 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:34,154 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791793971_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791793971_0001_00_000001, launchTime=1452791794154
2016-01-14 17:16:34,155 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791793971_0001_00_000001
2016-01-14 17:16:34,156 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791793971_0001_1_00_000000_0] to container: [container_1452791793971_0001_00_000001]
2016-01-14 17:16:34,156 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791793971_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:34,156 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791793971_0001_00_000001 given task: attempt_1452791793971_0001_1_00_000000_0
2016-01-14 17:16:34,156 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791793971_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791793971_0001_1_00_000000_0
2016-01-14 17:16:34,157 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791793971_0001_1_00_000000_0] started. Is using containerId: [container_1452791793971_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:34 Starting to run new task attempt: attempt_1452791793971_0001_1_00_000000_0
2016-01-14 17:16:34,157 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791793971_0001_1_00_000000_0
2016-01-14 17:16:34,158 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:34,158 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:34,158 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:34,158 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:34,158 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:34,159 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:34,159 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [BFF19B0CF0E2479593D6FB069DA351B5/D9444EA0E17B43E7B2F2119C2BC9739E] every/(1/1) ...latform/0_'field'_RESULTS, VertexName: A403C8FC1FCE4446837A056F98F9ACBF, VertexParallelism: 1, TaskAttemptID:attempt_1452791793971_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=2BAB98386D2D4BAA86715C87116A6E33, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=EFD18C74FFB34E38BB0B32FAB02F3C01, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:34,159 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=761790464
2016-01-14 17:16:34,158 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791793971_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A403C8FC1FCE4446837A056F98F9ACBF, taskAttemptId=attempt_1452791793971_0001_1_00_000000_0, startTime=1452791794156, containerId=container_1452791793971_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791793971_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:34,158 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:34,159 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791793971_0001_1_00_000000_0
2016-01-14 17:16:34,159 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:34,160 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:34,159 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791793971_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:34,160 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791793971_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:34,160 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:34,160 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791793971_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:34,161 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=2BAB98386D2D4BAA86715C87116A6E33, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:34,161 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:34,161 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 2BAB98386D2D4BAA86715C87116A6E33
2016-01-14 17:16:34,161 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 2BAB98386D2D4BAA86715C87116A6E33
2016-01-14 17:16:34,162 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:34,164 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:34,164 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 2BAB98386D2D4BAA86715C87116A6E33
2016-01-14 17:16:34,164 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 2BAB98386D2D4BAA86715C87116A6E33
2016-01-14 17:16:34,165 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=EFD18C74FFB34E38BB0B32FAB02F3C01, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:34,165 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:34,165 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: EFD18C74FFB34E38BB0B32FAB02F3C01
2016-01-14 17:16:34,167 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:34,167 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: EFD18C74FFB34E38BB0B32FAB02F3C01
2016-01-14 17:16:34,167 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: EFD18C74FFB34E38BB0B32FAB02F3C01
2016-01-14 17:16:34,167 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A403C8FC1FCE4446837A056F98F9ACBF, ordinal: 0
2016-01-14 17:16:34,167 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A403C8FC1FCE4446837A056F98F9ACBF, mem on start (mb), free: 288, total: 726, max: 726
2016-01-14 17:16:34,167 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:34,168 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:34,168 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:34,168 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:34,168 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:34,169 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:34,169 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:34,170 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:34,170 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:34,170 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 761790464, TotalAvailable: 510399610, TotalRequested/TotalJVMHeap:0.14
2016-01-14 17:16:34,171 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 2BAB98386D2D4BAA86715C87116A6E33, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:34,171 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, EFD18C74FFB34E38BB0B32FAB02F3C01, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:34,171 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:34,171 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 2BAB98386D2D4BAA86715C87116A6E33
2016-01-14 17:16:34,171 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:34,171 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 2BAB98386D2D4BAA86715C87116A6E33
2016-01-14 17:16:34,171 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 2BAB98386D2D4BAA86715C87116A6E33 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:34,171 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:34,171 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:34,171 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:34,173 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791793971_0001_1_00_000000_0
2016-01-14 17:16:34,175 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:34,176 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@6f359a46
2016-01-14 17:16:34,177 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:34,177 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 2BAB98386D2D4BAA86715C87116A6E33
2016-01-14 17:16:34,178 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 8C829DF97C454D88A7629F30688C606B
2016-01-14 17:16:34,178 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 8C829DF97C454D88A7629F30688C606B
2016-01-14 17:16:34,177 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@6f359a46
2016-01-14 17:16:34,178 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:34,179 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:34,179 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@325786cc; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@1052d4e6; comparator=cascading.tuple.hadoop.util.TupleComparator@2e175069; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:34,179 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:34,179 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:34,180 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:34,208 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:34,209 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:34,210 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:34,210 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:34,210 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:34,212 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:34,212 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:34,212 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A403C8FC1FCE4446837A056F98F9ACBF, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:34,215 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791793971_0001_1_00_000000_0
2016-01-14 17:16:34,215 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A403C8FC1FCE4446837A056F98F9ACBF, mem on close (mb), free: 386, total: 725, max: 725
2016-01-14 17:16:34,215 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:34,216 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:34,216 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:34,216 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:34,219 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:34 Completed running task attempt: attempt_1452791793971_0001_1_00_000000_0
2016-01-14 17:16:34,222 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791793971_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:34,225 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791793971_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791793971_0001_1_00, eventCount=1
2016-01-14 17:16:34,225 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=14, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=760741888, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791794212, Process_Duration=3, Process_End_Time=1452791794215, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=2][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:34,225 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:34,225 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:34,225 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:34,225 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791793971_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A403C8FC1FCE4446837A056F98F9ACBF, taskAttemptId=attempt_1452791793971_0001_1_00_000000_0, startTime=1452791794156, finishTime=1452791794225, timeTaken=69, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=14, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=760741888, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791794212, Process_Duration=3, Process_End_Time=1452791794215, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:34,226 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791793971_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:34,226 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791793971_0001_1_00_000000_0
2016-01-14 17:16:34,226 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:34,227 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791793971_0001_00_000001
2016-01-14 17:16:34,228 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791793971_0001_1][Event:TASK_FINISHED]: vertexName=A403C8FC1FCE4446837A056F98F9ACBF, taskId=task_1452791793971_0001_1_00_000000, startTime=1452791794156, finishTime=1452791794226, timeTaken=70, status=SUCCEEDED, successfulAttemptID=attempt_1452791793971_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=14, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=760741888, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791794212, Process_Duration=3, Process_End_Time=1452791794215, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:34,228 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791793971_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:34,228 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791793971_0001_1_00 [A403C8FC1FCE4446837A056F98F9ACBF] : 1
2016-01-14 17:16:34,228 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791793971_0001_1_00 [A403C8FC1FCE4446837A056F98F9ACBF]
2016-01-14 17:16:34,229 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791793971_0001_1][Event:VERTEX_FINISHED]: vertexName=A403C8FC1FCE4446837A056F98F9ACBF, vertexId=vertex_1452791793971_0001_1_00, initRequestedTime=1452791794132, initedTime=1452791794145, startRequestedTime=1452791794146, startedTime=1452791794146, finishTime=1452791794228, timeTaken=82, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=14, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=760741888, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791794212, Process_Duration=3, Process_End_Time=1452791794215, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791794156, firstTasksToStart=[ task_1452791793971_0001_1_00_000000 ], lastTaskFinishTime=1452791794225, lastTasksToFinish=[ task_1452791793971_0001_1_00_000000 ], minTaskDuration=69, maxTaskDuration=69, avgTaskDuration=69.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791793971_0001_1_00_000000 ], longestDurationTasks=[ task_1452791793971_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:34,229 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791793971_0001_1_00 [A403C8FC1FCE4446837A056F98F9ACBF] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:34,229 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791793971_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:34,230 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791793971_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791793971_0001_00_000001, stoppedTime=1452791794230, exitStatus=0
2016-01-14 17:16:34,230 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791793971_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:34,230 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791793971_0001_1_01 [EFD18C74FFB34E38BB0B32FAB02F3C01] attempt: attempt_1452791793971_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:34,230 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: EFD18C74FFB34E38BB0B32FAB02F3C01
2016-01-14 17:16:34,230 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791793971_0001_1_01 [EFD18C74FFB34E38BB0B32FAB02F3C01]
2016-01-14 17:16:34,230 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791793971_0001_1_00 [A403C8FC1FCE4446837A056F98F9ACBF] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:34,230 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791793971_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:34,230 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:34,231 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791793971_0001_1][Event:TASK_STARTED]: vertexName=EFD18C74FFB34E38BB0B32FAB02F3C01, taskId=task_1452791793971_0001_1_01_000000, scheduledTime=1452791794231, launchTime=1452791794231
2016-01-14 17:16:34,231 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791793971_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:34,231 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791793971_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:34,231 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:34,231 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791793971_0001_1_01 [EFD18C74FFB34E38BB0B32FAB02F3C01]
2016-01-14 17:16:34,231 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [BFF19B0CF0E2479593D6FB069DA351B5/D9444EA0E17B43E7B2F2119C2BC9739E] every/(1/1) ...latform/0_'field'_RESULTS, VertexName: EFD18C74FFB34E38BB0B32FAB02F3C01, VertexParallelism: 1, TaskAttemptID:attempt_1452791793971_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A403C8FC1FCE4446837A056F98F9ACBF, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=A00C5EEF886643E59D447868E0FBD3A0, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:34,231 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791793971_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:34,232 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:34,232 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:34,232 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:34,234 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791793971_0001_00_000002
2016-01-14 17:16:34,234 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791793971_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:34,236 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791793971_0001_00_000002
2016-01-14 17:16:34,236 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:34,237 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791793971_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:34,237 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:34,237 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791793971_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791793971_0001_00_000002, launchTime=1452791794237
2016-01-14 17:16:34,237 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791793971_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:34,238 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791793971_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:34,238 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791793971_0001_00_000001, Exiting
2016-01-14 17:16:34,238 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791793971_0001_00_000001
2016-01-14 17:16:34,238 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791793971_0001_00_000001
2016-01-14 17:16:34,239 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791793971_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:34,239 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791793971_0001_00_000001 completed successfully
2016-01-14 17:16:34,239 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791793971_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:34,239 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791793971_0001_00_000002
2016-01-14 17:16:34,240 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791793971_0001_1_01_000000_0] to container: [container_1452791793971_0001_00_000002]
2016-01-14 17:16:34,240 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791793971_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:34,241 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791793971_0001_1_01_000000_0] started. Is using containerId: [container_1452791793971_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:34,241 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:34,241 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791793971_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=EFD18C74FFB34E38BB0B32FAB02F3C01, taskAttemptId=attempt_1452791793971_0001_1_01_000000_0, startTime=1452791794240, containerId=container_1452791793971_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791793971_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:34,241 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791793971_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:34,241 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791793971_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:34,241 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791793971_0001_00_000002 given task: attempt_1452791793971_0001_1_01_000000_0
2016-01-14 17:16:34,242 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791793971_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791793971_0001_1_01_000000_0
2016-01-14 17:16:34 Starting to run new task attempt: attempt_1452791793971_0001_1_01_000000_0
2016-01-14 17:16:34,242 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791793971_0001_1_01_000000_0
2016-01-14 17:16:34,242 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:34,242 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:34,242 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:34,242 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:34,243 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:34,243 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:34,243 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [BFF19B0CF0E2479593D6FB069DA351B5/D9444EA0E17B43E7B2F2119C2BC9739E] every/(1/1) ...latform/0_'field'_RESULTS, VertexName: EFD18C74FFB34E38BB0B32FAB02F3C01, VertexParallelism: 1, TaskAttemptID:attempt_1452791793971_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A403C8FC1FCE4446837A056F98F9ACBF, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=A00C5EEF886643E59D447868E0FBD3A0, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:34,243 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=760741888
2016-01-14 17:16:34,243 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791793971_0001_1_01_000000_0
2016-01-14 17:16:34,244 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:34,244 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:34,244 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791793971_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:34,244 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=A403C8FC1FCE4446837A056F98F9ACBF, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:34,245 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:34,245 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: A403C8FC1FCE4446837A056F98F9ACBF
2016-01-14 17:16:34,245 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:34,245 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A00C5EEF886643E59D447868E0FBD3A0, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:34,246 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:34,246 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A00C5EEF886643E59D447868E0FBD3A0
2016-01-14 17:16:34,246 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:34,246 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: A00C5EEF886643E59D447868E0FBD3A0
2016-01-14 17:16:34,248 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:34,254 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 684667712, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:34,254 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:34,254 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 684667712, type: INPUT, componentVertexName: A403C8FC1FCE4446837A056F98F9ACBF
2016-01-14 17:16:34,255 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: A403C8FC1FCE4446837A056F98F9ACBF
2016-01-14 17:16:34,255 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: EFD18C74FFB34E38BB0B32FAB02F3C01, ordinal: 1
2016-01-14 17:16:34,255 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EFD18C74FFB34E38BB0B32FAB02F3C01, mem on start (mb), free: 380, total: 725, max: 725
2016-01-14 17:16:34,255 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:34,256 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:34,256 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:34,258 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:34,258 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A00C5EEF886643E59D447868E0FBD3A0
2016-01-14 17:16:34,258 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:34,258 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:34,258 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:34,258 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:34,259 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:34,259 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:34,259 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 684667712, TotalRequestedScaled: 6.32000964923077E8, TotalJVMHeap: 760741888, TotalAvailable: 509697064, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:34,259 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A00C5EEF886643E59D447868E0FBD3A0, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:34,259 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, A403C8FC1FCE4446837A056F98F9ACBF, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=684667712, allocated=509697064
2016-01-14 17:16:34,259 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:34,259 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: A403C8FC1FCE4446837A056F98F9ACBF being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:34,259 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:34,259 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:34,259 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: A403C8FC1FCE4446837A056F98F9ACBF
2016-01-14 17:16:34,260 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:34,261 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:34,261 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: A403C8FC1FCE4446837A056F98F9ACBF with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:34,261 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=684667712, postMergeMem=0, RuntimeTotalAvailable=509697064. Updated to: ShuffleMem=509697064, postMergeMem: 0
2016-01-14 17:16:34,261 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=509697064, maxSingleShuffleLimit=127424264, mergeThreshold=458727328, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:34,262 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:34,262 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: A403C8FC1FCE4446837A056F98F9ACBF
2016-01-14 17:16:34,263 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:34,263 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791793971_0001_1_01_000000_0
2016-01-14 17:16:34,263 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791793971_0001_1_00_000000_0_10129, runDuration: 55538, ]
2016-01-14 17:16:34,264 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791793971_0001_1_00_000000_0_10129] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791794264, TimeTaken=0, Rate=0.00 MB/s
2016-01-14 17:16:34,265 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : A403C8FC1FCE4446837A056F98F9ACBF
2016-01-14 17:16:34,265 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:34,265 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [A403C8FC1FCE4446837A056F98F9ACBF] #22 in 1ms
2016-01-14 17:16:34,265 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [A403C8FC1FCE4446837A056F98F9ACBF] #22, status:false, isInterrupted:false
2016-01-14 17:16:34,267 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:34,267 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 8C829DF97C454D88A7629F30688C606B
2016-01-14 17:16:34,267 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_RESULTS"], id: A00C5EEF886643E59D447868E0FBD3A0
2016-01-14 17:16:34,268 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_RESULTS"]
2016-01-14 17:16:34,268 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 8C829DF97C454D88A7629F30688C606B, for 1 inputs
2016-01-14 17:16:34,268 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:34,268 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:34,268 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:34,268 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:34,269 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:34,269 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : A403C8FC1FCE4446837A056F98F9ACBF
2016-01-14 17:16:34,269 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:34,269 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: EFD18C74FFB34E38BB0B32FAB02F3C01, all 1 inputs ready in: 00:00:00.001
2016-01-14 17:16:34,269 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@5f3fc4de; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@5395fbac
2016-01-14 17:16:34,274 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:34,274 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:34,274 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791793971_0001_1_01_000000_0
2016-01-14 17:16:34,274 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791793971_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:34,275 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791793971_0001_1_01_000000_0
2016-01-14 17:16:34,275 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EFD18C74FFB34E38BB0B32FAB02F3C01, mem on close (mb), free: 378, total: 725, max: 725
2016-01-14 17:16:34,275 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: A403C8FC1FCE4446837A056F98F9ACBF
2016-01-14 17:16:34,276 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:34,276 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:34,277 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:34,277 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:34,277 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:34 Completed running task attempt: attempt_1452791793971_0001_1_01_000000_0
2016-01-14 17:16:34,278 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791793971_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:34,278 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791793971_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=EFD18C74FFB34E38BB0B32FAB02F3C01, taskAttemptId=attempt_1452791793971_0001_1_01_000000_0, startTime=1452791794240, finishTime=1452791794278, timeTaken=38, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=760741888, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791794269, Process_Duration=6, Process_End_Time=1452791794275, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:34,279 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791793971_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:34,279 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791793971_0001_1_01_000000_0
2016-01-14 17:16:34,279 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:34,279 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791793971_0001_1][Event:TASK_FINISHED]: vertexName=EFD18C74FFB34E38BB0B32FAB02F3C01, taskId=task_1452791793971_0001_1_01_000000, startTime=1452791794240, finishTime=1452791794279, timeTaken=39, status=SUCCEEDED, successfulAttemptID=attempt_1452791793971_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=760741888, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791794269, Process_Duration=6, Process_End_Time=1452791794275, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:34,280 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791793971_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:34,280 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791793971_0001_1_01 [EFD18C74FFB34E38BB0B32FAB02F3C01] : 1
2016-01-14 17:16:34,280 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791793971_0001_1_01 [EFD18C74FFB34E38BB0B32FAB02F3C01]
2016-01-14 17:16:34,280 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=760741888, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791794269, Process_Duration=6, Process_End_Time=1452791794275, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 17:16:34,280 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:34,280 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:34,281 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791793971_0001_00_000002
2016-01-14 17:16:34,281 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:34,281 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791793971_0001_1][Event:VERTEX_FINISHED]: vertexName=EFD18C74FFB34E38BB0B32FAB02F3C01, vertexId=vertex_1452791793971_0001_1_01, initRequestedTime=1452791794134, initedTime=1452791794142, startRequestedTime=1452791794148, startedTime=1452791794148, finishTime=1452791794280, timeTaken=132, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=760741888, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791794269, Process_Duration=6, Process_End_Time=1452791794275, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452791794240, firstTasksToStart=[ task_1452791793971_0001_1_01_000000 ], lastTaskFinishTime=1452791794278, lastTasksToFinish=[ task_1452791793971_0001_1_01_000000 ], minTaskDuration=38, maxTaskDuration=38, avgTaskDuration=38.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791793971_0001_1_01_000000 ], longestDurationTasks=[ task_1452791793971_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:34,281 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791793971_0001_1_01 [EFD18C74FFB34E38BB0B32FAB02F3C01] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:34,282 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791793971_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:34,282 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791793971_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791793971_0001_00_000002, stoppedTime=1452791794282, exitStatus=0
2016-01-14 17:16:34,282 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791793971_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:34,282 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791793971_0001_1_01 [EFD18C74FFB34E38BB0B32FAB02F3C01] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:34,282 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:34,283 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791793971_0001_1
2016-01-14 17:16:34,283 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:34,283 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791793971_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452791793971_0001_1
2016-01-14 17:16:34,284 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452791793971_0001_1_00 [A403C8FC1FCE4446837A056F98F9ACBF]
2016-01-14 17:16:34,284 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: A00C5EEF886643E59D447868E0FBD3A0 for vertex: vertex_1452791793971_0001_1_01 [EFD18C74FFB34E38BB0B32FAB02F3C01]
2016-01-14 17:16:34,284 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:34,284 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791793971_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 17:16:34,285 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791793971_0001_00_000002: 1 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:34,285 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791793971_0001_00_000002, Exiting
2016-01-14 17:16:34,285 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791793971_0001_00_000002
2016-01-14 17:16:34,285 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791793971_0001_00_000002
2016-01-14 17:16:34,285 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791793971_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:34,285 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791793971_0001_00_000002 completed successfully
2016-01-14 17:16:34,294 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791793971_0001_1, queueSize=0
2016-01-14 17:16:34,294 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:34,294 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791793971_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791793971_0001_1, startTime=1452791794132, finishTime=1452791794292, timeTaken=160, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=985, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=14, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1521483776, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905583588481, Process_Duration=9, Process_End_Time=2905583588490, Read_Duration=0, Tuples_Read=40, Tuples_Written=40, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 17:16:34,294 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791793971_0001_1 finished with state: SUCCEEDED
2016-01-14 17:16:34,294 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791793971_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:34,294 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452791793971_0001_00_000002
2016-01-14 17:16:34,295 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791793971_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:16:34 Completed Dag: dag_1452791793971_0001_1
2016-01-14 17:16:34,295 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791793971_0001_1, dagState=SUCCEEDED
2016-01-14 17:16:34,295 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791793971_0001_1_post
2016-01-14 17:16:34,296 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:34,296 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:34,300 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/0_'field'_RESULTS, applicationId=application_1452791793971_0001
2016-01-14 17:16:34,302 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:34,302 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:34,302 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:34,302 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:34,302 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:34,303 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:34,303 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:34,303 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:34,304 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:34,304 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:34,304 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:34,304 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:34,304 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:34,309 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/BFF19B0CF0E2479593D6FB069DA351B5/D9444EA0E17B43E7B2F2119C2BC9739E/.tez/application_1452791793971_0001
2016-01-14 17:16:34,309 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.358
2016-01-14 17:16:34,378 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_'field'_RESULTS
2016-01-14 17:16:34,401 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:34,402 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:34,437 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.011
2016-01-14 17:16:34,441 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.016
2016-01-14 17:16:34,441 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:34,442 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:34,442 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:34,458 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:34,464 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_ALL
2016-01-14 17:16:34,510 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:34,510 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:34,510 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_ALL"]
2016-01-14 17:16:34,510 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:34,510 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:34,510 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:34,511 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...blyplatform/0_'field'_ALL
2016-01-14 17:16:34,511 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:34,523 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:34,527 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:34,527 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:34,527 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:34,536 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E53FD299CF414C82B836115DDC358FC6/C7FD7154468C48C080D9E1A1829274B4/.tez/application_1452791794527_0001 doesn't exist and is created
2016-01-14 17:16:34,553 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:34,554 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:34,554 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E53FD299CF414C82B836115DDC358FC6/C7FD7154468C48C080D9E1A1829274B4/.tez/application_1452791794527_0001
2016-01-14 17:16:34,558 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791794527_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:34,568 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:34,569 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:34,570 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:34,570 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:34,571 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E53FD299CF414C82B836115DDC358FC6/C7FD7154468C48C080D9E1A1829274B4/.tez/application_1452791794527_0001/localmode-log-dir
2016-01-14 17:16:34,571 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E53FD299CF414C82B836115DDC358FC6/C7FD7154468C48C080D9E1A1829274B4/.tez/application_1452791794527_0001/localmode-log-dir/history.txt.appattempt_1452791794527_0001_000000, maxErrors=10
2016-01-14 17:16:34,571 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:34,571 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791794527_0001_000000, appSubmitTime=1452791794558, launchTime=1452791794558
2016-01-14 17:16:34,573 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:34,576 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:34,577 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:34,577 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:34,579 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:48505
2016-01-14 17:16:34,582 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:34,583 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 17:16:34,583 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 17:16:34,583 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791794527_0001_000000, startTime=1452791794583
2016-01-14 17:16:34,584 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:34,584 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:34,654 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:34,654 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:34,655 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...blyplatform/0_'field'_ALL, applicationId=application_1452791794527_0001, dagName=[E53FD299CF414C82B836115DDC358FC6/C7FD7154468C48C080D9E1A1829274B4] every/(1/1) ...blyplatform/0_'field'_ALL
2016-01-14 17:16:34,669 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [E53FD299CF414C82B836115DDC358FC6/C7FD7154468C48C080D9E1A1829274B4] every/(1/1) ...blyplatform/0_'field'_ALL
2016-01-14 17:16:34,671 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791794527_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E53FD299CF414C82B836115DDC358FC6/C7FD7154468C48C080D9E1A1829274B4/.tez/application_1452791794527_0001/localmode-log-dir/dag_1452791794527_0001_1.dot
2016-01-14 17:16:34,671 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E53FD299CF414C82B836115DDC358FC6/C7FD7154468C48C080D9E1A1829274B4/.tez/application_1452791794527_0001/localmode-log-dir/dag_1452791794527_0001_1-tez-dag.pb.txt
2016-01-14 17:16:34,677 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791794527_0001_1
2016-01-14 17:16:34,677 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:34,677 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [E53FD299CF414C82B836115DDC358FC6/C7FD7154468C48C080D9E1A1829274B4] every/(1/1) ...blyplatform/0_'field'_ALL
2016-01-14 17:16:34 Running Dag: dag_1452791794527_0001_1
2016-01-14 17:16:34,689 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791794527_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791794527_0001_1, submitTime=1452791794670
2016-01-14 17:16:34,689 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:34,690 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex B84A997CD95E4DCF909FF20A5A34F13C
2016-01-14 17:16:34,690 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:34,691 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:34,691 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791794527_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791794527_0001_1, initTime=1452791794690
2016-01-14 17:16:34,691 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791794527_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:34,692 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791794527_0001_1][Event:DAG_STARTED]: dagID=dag_1452791794527_0001_1, startTime=1452791794691
2016-01-14 17:16:34,692 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:34,692 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:34,692 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...blyplatform/0_'field'_ALL, applicationId=application_1452791794527_0001, dagName=[E53FD299CF414C82B836115DDC358FC6/C7FD7154468C48C080D9E1A1829274B4] every/(1/1) ...blyplatform/0_'field'_ALL
2016-01-14 17:16:34,692 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791794527_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:34,692 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: F736F3D3B39E48818FBA35913AC7CB27 : {9F6BF97905D1439CBCDA6E99F5638895={InputName=9F6BF97905D1439CBCDA6E99F5638895}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:34,692 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 9F6BF97905D1439CBCDA6E99F5638895, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:34,692 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791794527_0001_1_00 [F736F3D3B39E48818FBA35913AC7CB27]
2016-01-14 17:16:34,692 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791794527_0001_1_00 [F736F3D3B39E48818FBA35913AC7CB27]
2016-01-14 17:16:34,693 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791794527_0001_1_00 [F736F3D3B39E48818FBA35913AC7CB27]
2016-01-14 17:16:34,693 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791794527_0001_1_00 [F736F3D3B39E48818FBA35913AC7CB27]. Starting root input initializers: 1
2016-01-14 17:16:34,693 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791794527_0001_1_00 [F736F3D3B39E48818FBA35913AC7CB27] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:34,693 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791794527_0001_1_01 [B84A997CD95E4DCF909FF20A5A34F13C]
2016-01-14 17:16:34,693 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 9F6BF97905D1439CBCDA6E99F5638895 on vertex vertex_1452791794527_0001_1_00 [F736F3D3B39E48818FBA35913AC7CB27]
2016-01-14 17:16:34,695 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:34,695 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 9F6BF97905D1439CBCDA6E99F5638895 asking for 1 tasks. Headroom: 725 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:34,696 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:34,696 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:34,696 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:34,697 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791794527_0001_1_01 [B84A997CD95E4DCF909FF20A5A34F13C]
2016-01-14 17:16:34,697 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791794527_0001_1_01 [B84A997CD95E4DCF909FF20A5A34F13C]
2016-01-14 17:16:34,697 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791794527_0001_1_01 [B84A997CD95E4DCF909FF20A5A34F13C]
2016-01-14 17:16:34,697 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=0013EE4AC3E44A50990E95E89A217771, vertexId=vertex_1452791794527_0001_1_01 [B84A997CD95E4DCF909FF20A5A34F13C], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:34,697 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=0013EE4AC3E44A50990E95E89A217771, vertexId=vertex_1452791794527_0001_1_01 [B84A997CD95E4DCF909FF20A5A34F13C]
2016-01-14 17:16:34,699 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:34,699 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 9F6BF97905D1439CBCDA6E99F5638895 on vertex vertex_1452791794527_0001_1_00 [F736F3D3B39E48818FBA35913AC7CB27]
2016-01-14 17:16:34,699 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=0013EE4AC3E44A50990E95E89A217771, vertexName=B84A997CD95E4DCF909FF20A5A34F13C, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:34,700 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=0013EE4AC3E44A50990E95E89A217771, vertexId=vertex_1452791794527_0001_1_01 [B84A997CD95E4DCF909FF20A5A34F13C]
2016-01-14 17:16:34,701 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:34,701 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791794527_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B84A997CD95E4DCF909FF20A5A34F13C, vertexId=vertex_1452791794527_0001_1_01, initRequestedTime=1452791794693, initedTime=1452791794700, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:34,701 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791794527_0001_1_01 [B84A997CD95E4DCF909FF20A5A34F13C] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:34,701 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791794527_0001_1_00 [F736F3D3B39E48818FBA35913AC7CB27] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:34,701 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {9F6BF97905D1439CBCDA6E99F5638895=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:34,702 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791794527_0001_1_00 [F736F3D3B39E48818FBA35913AC7CB27] parallelism set to 1
2016-01-14 17:16:34,702 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791794527_0001_1_00 [F736F3D3B39E48818FBA35913AC7CB27] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:34,702 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791794527_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791794527_0001_1_00, eventCount=1
2016-01-14 17:16:34,702 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:34,703 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791794527_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F736F3D3B39E48818FBA35913AC7CB27, vertexId=vertex_1452791794527_0001_1_00, initRequestedTime=1452791794692, initedTime=1452791794702, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:34,703 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791794527_0001_1_00 [F736F3D3B39E48818FBA35913AC7CB27] with distanceFromRoot: 0
2016-01-14 17:16:34,703 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791794527_0001_1_00 [F736F3D3B39E48818FBA35913AC7CB27] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:34,703 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in F736F3D3B39E48818FBA35913AC7CB27
2016-01-14 17:16:34,703 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791794527_0001_1_00 [F736F3D3B39E48818FBA35913AC7CB27]
2016-01-14 17:16:34,704 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:34,704 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791794527_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791794527_0001_1_00, startRequestedTime=1452791794703, startedTime=1452791794703
2016-01-14 17:16:34,704 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791794527_0001_1_00 [F736F3D3B39E48818FBA35913AC7CB27] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:34,704 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:34,705 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791794527_0001_1][Event:TASK_STARTED]: vertexName=F736F3D3B39E48818FBA35913AC7CB27, taskId=task_1452791794527_0001_1_00_000000, scheduledTime=1452791794704, launchTime=1452791794704
2016-01-14 17:16:34,705 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791794527_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:34,705 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791794527_0001_1_00 for vertex: vertex_1452791794527_0001_1_01 [B84A997CD95E4DCF909FF20A5A34F13C] numStartedSources: 1 numSources: 1
2016-01-14 17:16:34,705 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791794527_0001_1_01 [B84A997CD95E4DCF909FF20A5A34F13C] with distanceFromRoot: 1
2016-01-14 17:16:34,705 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791794527_0001_1_00 [F736F3D3B39E48818FBA35913AC7CB27]
2016-01-14 17:16:34,706 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: F736F3D3B39E48818FBA35913AC7CB27 in vertex: B84A997CD95E4DCF909FF20A5A34F13C
2016-01-14 17:16:34,706 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: B84A997CD95E4DCF909FF20A5A34F13C with 1 source tasks and 1 pending tasks
2016-01-14 17:16:34,706 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:34,706 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791794527_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791794527_0001_1_01, startRequestedTime=1452791794705, startedTime=1452791794705
2016-01-14 17:16:34,706 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791794527_0001_1_01 [B84A997CD95E4DCF909FF20A5A34F13C] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:34,707 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [E53FD299CF414C82B836115DDC358FC6/C7FD7154468C48C080D9E1A1829274B4] every/(1/1) ...blyplatform/0_'field'_ALL, VertexName: F736F3D3B39E48818FBA35913AC7CB27, VertexParallelism: 1, TaskAttemptID:attempt_1452791794527_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=9F6BF97905D1439CBCDA6E99F5638895, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B84A997CD95E4DCF909FF20A5A34F13C, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:34,707 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791794527_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:34,707 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:34,708 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:34,708 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:34,708 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:34,709 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791794527_0001_00_000001
2016-01-14 17:16:34,709 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791794527_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:34,709 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791794527_0001, with dag id: dag_1452791794527_0001_1
2016-01-14 17:16:34,710 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791794527_0001_00_000001
2016-01-14 17:16:34,710 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:34,711 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791794527_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:34,711 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:34,711 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791794527_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791794527_0001_00_000001, launchTime=1452791794711
2016-01-14 17:16:34,712 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791794527_0001_00_000001
2016-01-14 17:16:34,712 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791794527_0001_1_00_000000_0] to container: [container_1452791794527_0001_00_000001]
2016-01-14 17:16:34,713 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791794527_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:34,713 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791794527_0001_1_00_000000_0] started. Is using containerId: [container_1452791794527_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:34,713 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791794527_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F736F3D3B39E48818FBA35913AC7CB27, taskAttemptId=attempt_1452791794527_0001_1_00_000000_0, startTime=1452791794713, containerId=container_1452791794527_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791794527_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:34,713 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791794527_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:34,713 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791794527_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:34,714 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:34,714 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791794527_0001_00_000001 given task: attempt_1452791794527_0001_1_00_000000_0
2016-01-14 17:16:34,714 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791794527_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791794527_0001_1_00_000000_0
2016-01-14 17:16:34 Starting to run new task attempt: attempt_1452791794527_0001_1_00_000000_0
2016-01-14 17:16:34,714 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791794527_0001_1_00_000000_0
2016-01-14 17:16:34,715 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:34,715 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:34,715 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:34,715 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:34,715 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:34,715 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:34,715 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [E53FD299CF414C82B836115DDC358FC6/C7FD7154468C48C080D9E1A1829274B4] every/(1/1) ...blyplatform/0_'field'_ALL, VertexName: F736F3D3B39E48818FBA35913AC7CB27, VertexParallelism: 1, TaskAttemptID:attempt_1452791794527_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=9F6BF97905D1439CBCDA6E99F5638895, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B84A997CD95E4DCF909FF20A5A34F13C, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:34,715 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=760741888
2016-01-14 17:16:34,715 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791794527_0001_1_00_000000_0
2016-01-14 17:16:34,716 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:34,716 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:34,716 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:34,717 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=9F6BF97905D1439CBCDA6E99F5638895, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:34,717 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791794527_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:34,717 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:34,717 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 9F6BF97905D1439CBCDA6E99F5638895
2016-01-14 17:16:34,717 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 9F6BF97905D1439CBCDA6E99F5638895
2016-01-14 17:16:34,718 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=B84A997CD95E4DCF909FF20A5A34F13C, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:34,719 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:34,719 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: B84A997CD95E4DCF909FF20A5A34F13C
2016-01-14 17:16:34,719 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:34,720 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:34,720 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 9F6BF97905D1439CBCDA6E99F5638895
2016-01-14 17:16:34,720 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 9F6BF97905D1439CBCDA6E99F5638895
2016-01-14 17:16:34,721 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:34,721 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: B84A997CD95E4DCF909FF20A5A34F13C
2016-01-14 17:16:34,721 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: B84A997CD95E4DCF909FF20A5A34F13C
2016-01-14 17:16:34,727 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F736F3D3B39E48818FBA35913AC7CB27, ordinal: 0
2016-01-14 17:16:34,727 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F736F3D3B39E48818FBA35913AC7CB27, mem on start (mb), free: 286, total: 725, max: 725
2016-01-14 17:16:34,727 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:34,728 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:34,728 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:34,728 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:34,728 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:34,728 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:34,728 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:34,728 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:34,728 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:34,729 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 760741888, TotalAvailable: 509697064, TotalRequested/TotalJVMHeap:0.14
2016-01-14 17:16:34,729 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 9F6BF97905D1439CBCDA6E99F5638895, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:34,729 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, B84A997CD95E4DCF909FF20A5A34F13C, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:34,729 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:34,729 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 9F6BF97905D1439CBCDA6E99F5638895 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:34,730 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:34,730 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:34,729 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 9F6BF97905D1439CBCDA6E99F5638895
2016-01-14 17:16:34,730 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:34,730 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 9F6BF97905D1439CBCDA6E99F5638895
2016-01-14 17:16:34,730 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:34,731 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791794527_0001_1_00_000000_0
2016-01-14 17:16:34,731 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:34,731 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@72a5948f
2016-01-14 17:16:34,732 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@72a5948f
2016-01-14 17:16:34,732 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:34,732 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:34,734 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:34,735 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 9F6BF97905D1439CBCDA6E99F5638895
2016-01-14 17:16:34,735 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: E42D74A576E84A6EA3363BA974514A83
2016-01-14 17:16:34,735 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] E42D74A576E84A6EA3363BA974514A83
2016-01-14 17:16:34,736 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@433a32ea; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@21e4c321; comparator=cascading.tuple.hadoop.util.TupleComparator@31c6bce9; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:34,736 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:34,736 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:34,736 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:34,766 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:34,766 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:34,767 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:34,768 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:34,768 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:34,769 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:34,769 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:34,769 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F736F3D3B39E48818FBA35913AC7CB27, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:34,772 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791794527_0001_1_00_000000_0
2016-01-14 17:16:34,772 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F736F3D3B39E48818FBA35913AC7CB27, mem on close (mb), free: 382, total: 723, max: 723
2016-01-14 17:16:34,772 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:34,772 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:34,773 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:34,773 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:34,776 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:34 Completed running task attempt: attempt_1452791794527_0001_1_00_000000_0
2016-01-14 17:16:34,781 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791794527_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:34,781 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=16, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=758120448, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791794770, Process_Duration=2, Process_End_Time=1452791794772, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:34,781 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791794527_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791794527_0001_1_00, eventCount=1
2016-01-14 17:16:34,782 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:34,782 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791794527_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F736F3D3B39E48818FBA35913AC7CB27, taskAttemptId=attempt_1452791794527_0001_1_00_000000_0, startTime=1452791794713, finishTime=1452791794782, timeTaken=69, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=16, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=758120448, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791794770, Process_Duration=2, Process_End_Time=1452791794772, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:34,782 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791794527_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:34,782 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791794527_0001_1_00_000000_0
2016-01-14 17:16:34,782 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:34,784 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:34,784 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:34,784 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791794527_0001_00_000001
2016-01-14 17:16:34,784 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791794527_0001_1][Event:TASK_FINISHED]: vertexName=F736F3D3B39E48818FBA35913AC7CB27, taskId=task_1452791794527_0001_1_00_000000, startTime=1452791794713, finishTime=1452791794782, timeTaken=69, status=SUCCEEDED, successfulAttemptID=attempt_1452791794527_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=16, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=758120448, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791794770, Process_Duration=2, Process_End_Time=1452791794772, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:34,784 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791794527_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:34,784 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791794527_0001_1_00 [F736F3D3B39E48818FBA35913AC7CB27] : 1
2016-01-14 17:16:34,785 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791794527_0001_1_00 [F736F3D3B39E48818FBA35913AC7CB27]
2016-01-14 17:16:34,785 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791794527_0001_1][Event:VERTEX_FINISHED]: vertexName=F736F3D3B39E48818FBA35913AC7CB27, vertexId=vertex_1452791794527_0001_1_00, initRequestedTime=1452791794692, initedTime=1452791794702, startRequestedTime=1452791794703, startedTime=1452791794703, finishTime=1452791794785, timeTaken=82, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=16, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=758120448, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791794770, Process_Duration=2, Process_End_Time=1452791794772, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791794713, firstTasksToStart=[ task_1452791794527_0001_1_00_000000 ], lastTaskFinishTime=1452791794782, lastTasksToFinish=[ task_1452791794527_0001_1_00_000000 ], minTaskDuration=69, maxTaskDuration=69, avgTaskDuration=69.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791794527_0001_1_00_000000 ], longestDurationTasks=[ task_1452791794527_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:34,785 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791794527_0001_1_00 [F736F3D3B39E48818FBA35913AC7CB27] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:34,785 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791794527_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:34,785 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791794527_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791794527_0001_00_000001, stoppedTime=1452791794785, exitStatus=0
2016-01-14 17:16:34,785 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791794527_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:34,786 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791794527_0001_1_01 [B84A997CD95E4DCF909FF20A5A34F13C] attempt: attempt_1452791794527_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:34,786 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: B84A997CD95E4DCF909FF20A5A34F13C
2016-01-14 17:16:34,786 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791794527_0001_1_01 [B84A997CD95E4DCF909FF20A5A34F13C]
2016-01-14 17:16:34,786 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791794527_0001_1_00 [F736F3D3B39E48818FBA35913AC7CB27] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:34,786 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791794527_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:34,786 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791794527_0001_1][Event:TASK_STARTED]: vertexName=B84A997CD95E4DCF909FF20A5A34F13C, taskId=task_1452791794527_0001_1_01_000000, scheduledTime=1452791794786, launchTime=1452791794786
2016-01-14 17:16:34,786 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791794527_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:34,786 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791794527_0001_1_01 [B84A997CD95E4DCF909FF20A5A34F13C]
2016-01-14 17:16:34,786 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:34,787 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:34,787 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:34,787 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:34,787 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791794527_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:34,788 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [E53FD299CF414C82B836115DDC358FC6/C7FD7154468C48C080D9E1A1829274B4] every/(1/1) ...blyplatform/0_'field'_ALL, VertexName: B84A997CD95E4DCF909FF20A5A34F13C, VertexParallelism: 1, TaskAttemptID:attempt_1452791794527_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F736F3D3B39E48818FBA35913AC7CB27, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=0013EE4AC3E44A50990E95E89A217771, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:34,788 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791794527_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:34,788 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:34,789 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791794527_0001_00_000002
2016-01-14 17:16:34,789 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791794527_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:34,790 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791794527_0001_00_000002
2016-01-14 17:16:34,790 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:34,791 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791794527_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:34,791 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791794527_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791794527_0001_00_000002, launchTime=1452791794791
2016-01-14 17:16:34,791 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:34,793 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791794527_0001_00_000002
2016-01-14 17:16:34,794 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791794527_0001_1_01_000000_0] to container: [container_1452791794527_0001_00_000002]
2016-01-14 17:16:34,794 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791794527_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:34,794 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791794527_0001_00_000002 given task: attempt_1452791794527_0001_1_01_000000_0
2016-01-14 17:16:34,794 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791794527_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791794527_0001_1_01_000000_0
2016-01-14 17:16:34,794 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791794527_0001_1_01_000000_0] started. Is using containerId: [container_1452791794527_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:34 Starting to run new task attempt: attempt_1452791794527_0001_1_01_000000_0
2016-01-14 17:16:34,794 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791794527_0001_1_01_000000_0
2016-01-14 17:16:34,794 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:34,794 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791794527_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B84A997CD95E4DCF909FF20A5A34F13C, taskAttemptId=attempt_1452791794527_0001_1_01_000000_0, startTime=1452791794794, containerId=container_1452791794527_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791794527_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:34,795 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:34,795 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791794527_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:34,795 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791794527_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:34,795 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791794527_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:34,795 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791794527_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:34,795 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791794527_0001_00_000001, Exiting
2016-01-14 17:16:34,795 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791794527_0001_00_000001
2016-01-14 17:16:34,796 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791794527_0001_00_000001
2016-01-14 17:16:34,796 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:34,796 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:34,796 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:34,796 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:34,797 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:34,797 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791794527_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:34,797 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791794527_0001_00_000001 completed successfully
2016-01-14 17:16:34,797 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [E53FD299CF414C82B836115DDC358FC6/C7FD7154468C48C080D9E1A1829274B4] every/(1/1) ...blyplatform/0_'field'_ALL, VertexName: B84A997CD95E4DCF909FF20A5A34F13C, VertexParallelism: 1, TaskAttemptID:attempt_1452791794527_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F736F3D3B39E48818FBA35913AC7CB27, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=0013EE4AC3E44A50990E95E89A217771, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:34,797 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791794527_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:34,798 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=758120448
2016-01-14 17:16:34,798 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791794527_0001_1_01_000000_0
2016-01-14 17:16:34,798 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:34,798 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:34,798 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791794527_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:34,799 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:34,799 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F736F3D3B39E48818FBA35913AC7CB27, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:34,799 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=0013EE4AC3E44A50990E95E89A217771, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:34,800 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:34,800 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:34,800 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F736F3D3B39E48818FBA35913AC7CB27
2016-01-14 17:16:34,800 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 0013EE4AC3E44A50990E95E89A217771
2016-01-14 17:16:34,801 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:34,801 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 0013EE4AC3E44A50990E95E89A217771
2016-01-14 17:16:34,802 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:34,802 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 682308416, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:34,803 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:34,803 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 682308416, type: INPUT, componentVertexName: F736F3D3B39E48818FBA35913AC7CB27
2016-01-14 17:16:34,803 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F736F3D3B39E48818FBA35913AC7CB27
2016-01-14 17:16:34,809 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B84A997CD95E4DCF909FF20A5A34F13C, ordinal: 1
2016-01-14 17:16:34,809 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B84A997CD95E4DCF909FF20A5A34F13C, mem on start (mb), free: 376, total: 723, max: 723
2016-01-14 17:16:34,809 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:34,809 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:34,810 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:34,813 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:34,813 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 0013EE4AC3E44A50990E95E89A217771
2016-01-14 17:16:34,813 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:34,814 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:34,814 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:34,814 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:34,814 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:34,814 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:34,815 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 682308416, TotalRequestedScaled: 6.298231532307693E8, TotalJVMHeap: 758120448, TotalAvailable: 507940700, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:34,815 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 0013EE4AC3E44A50990E95E89A217771, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:34,815 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F736F3D3B39E48818FBA35913AC7CB27, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=682308416, allocated=507940700
2016-01-14 17:16:34,815 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:34,815 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F736F3D3B39E48818FBA35913AC7CB27 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:34,815 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F736F3D3B39E48818FBA35913AC7CB27
2016-01-14 17:16:34,816 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:34,816 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:34,816 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:34,817 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:34,817 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: F736F3D3B39E48818FBA35913AC7CB27 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:34,817 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=682308416, postMergeMem=0, RuntimeTotalAvailable=507940700. Updated to: ShuffleMem=507940700, postMergeMem: 0
2016-01-14 17:16:34,817 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=507940700, maxSingleShuffleLimit=126985176, mergeThreshold=457146624, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:34,817 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:34,820 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F736F3D3B39E48818FBA35913AC7CB27
2016-01-14 17:16:34,820 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:34,820 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791794527_0001_1_01_000000_0
2016-01-14 17:16:34,822 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791794527_0001_1_00_000000_0_10135, runDuration: 59332, ]
2016-01-14 17:16:34,823 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791794527_0001_1_00_000000_0_10135] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791794823, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 17:16:34,823 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : F736F3D3B39E48818FBA35913AC7CB27
2016-01-14 17:16:34,823 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:34,824 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:34,824 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [F736F3D3B39E48818FBA35913AC7CB27] #23 in 2ms
2016-01-14 17:16:34,824 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: E42D74A576E84A6EA3363BA974514A83
2016-01-14 17:16:34,825 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [F736F3D3B39E48818FBA35913AC7CB27] #23, status:false, isInterrupted:false
2016-01-14 17:16:34,825 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:34,825 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:34,826 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:34,826 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:34,826 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_ALL"], id: 0013EE4AC3E44A50990E95E89A217771
2016-01-14 17:16:34,826 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:34,826 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_ALL"]
2016-01-14 17:16:34,826 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : F736F3D3B39E48818FBA35913AC7CB27
2016-01-14 17:16:34,826 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] E42D74A576E84A6EA3363BA974514A83, for 1 inputs
2016-01-14 17:16:34,826 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:34,827 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B84A997CD95E4DCF909FF20A5A34F13C, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:34,827 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@8b859b3; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@7f08fb3d
2016-01-14 17:16:34,833 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:34,833 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:34,834 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791794527_0001_1_01_000000_0
2016-01-14 17:16:34,834 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791794527_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:34,835 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791794527_0001_1_01_000000_0
2016-01-14 17:16:34,835 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B84A997CD95E4DCF909FF20A5A34F13C, mem on close (mb), free: 374, total: 723, max: 723
2016-01-14 17:16:34,835 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: F736F3D3B39E48818FBA35913AC7CB27
2016-01-14 17:16:34,835 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:34,835 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:34,836 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:34,836 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:34,836 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:34 Completed running task attempt: attempt_1452791794527_0001_1_01_000000_0
2016-01-14 17:16:34,837 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791794527_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:34,837 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=758120448, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791794827, Process_Duration=8, Process_End_Time=1452791794835, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 17:16:34,837 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:34,837 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:34,838 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:34,837 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791794527_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B84A997CD95E4DCF909FF20A5A34F13C, taskAttemptId=attempt_1452791794527_0001_1_01_000000_0, startTime=1452791794794, finishTime=1452791794837, timeTaken=43, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=758120448, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791794827, Process_Duration=8, Process_End_Time=1452791794835, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:34,838 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791794527_0001_00_000002
2016-01-14 17:16:34,838 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791794527_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:34,839 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791794527_0001_1_01_000000_0
2016-01-14 17:16:34,839 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:34,839 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:34,839 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791794527_0001_1][Event:TASK_FINISHED]: vertexName=B84A997CD95E4DCF909FF20A5A34F13C, taskId=task_1452791794527_0001_1_01_000000, startTime=1452791794794, finishTime=1452791794839, timeTaken=45, status=SUCCEEDED, successfulAttemptID=attempt_1452791794527_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=758120448, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791794827, Process_Duration=8, Process_End_Time=1452791794835, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:34,839 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791794527_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:34,839 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791794527_0001_1_01 [B84A997CD95E4DCF909FF20A5A34F13C] : 1
2016-01-14 17:16:34,839 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791794527_0001_1_01 [B84A997CD95E4DCF909FF20A5A34F13C]
2016-01-14 17:16:34,840 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791794527_0001_1][Event:VERTEX_FINISHED]: vertexName=B84A997CD95E4DCF909FF20A5A34F13C, vertexId=vertex_1452791794527_0001_1_01, initRequestedTime=1452791794693, initedTime=1452791794700, startRequestedTime=1452791794705, startedTime=1452791794705, finishTime=1452791794839, timeTaken=134, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=758120448, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791794827, Process_Duration=8, Process_End_Time=1452791794835, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452791794794, firstTasksToStart=[ task_1452791794527_0001_1_01_000000 ], lastTaskFinishTime=1452791794837, lastTasksToFinish=[ task_1452791794527_0001_1_01_000000 ], minTaskDuration=43, maxTaskDuration=43, avgTaskDuration=43.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791794527_0001_1_01_000000 ], longestDurationTasks=[ task_1452791794527_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:34,840 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791794527_0001_1_01 [B84A997CD95E4DCF909FF20A5A34F13C] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:34,840 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:34,840 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791794527_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:34,841 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791794527_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791794527_0001_00_000002, stoppedTime=1452791794841, exitStatus=0
2016-01-14 17:16:34,841 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791794527_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:34,841 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791794527_0001_1_01 [B84A997CD95E4DCF909FF20A5A34F13C] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:34,841 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791794527_0001_1
2016-01-14 17:16:34,841 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791794527_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452791794527_0001_1
2016-01-14 17:16:34,841 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 0013EE4AC3E44A50990E95E89A217771 for vertex: vertex_1452791794527_0001_1_01 [B84A997CD95E4DCF909FF20A5A34F13C]
2016-01-14 17:16:34,841 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:34,849 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791794527_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 17:16:34,849 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791794527_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:34,849 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791794527_0001_00_000002, Exiting
2016-01-14 17:16:34,849 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791794527_0001_00_000002
2016-01-14 17:16:34,849 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791794527_0001_00_000002
2016-01-14 17:16:34,849 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791794527_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:34,850 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791794527_0001_00_000002 completed successfully
2016-01-14 17:16:34,850 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452791794527_0001_1_00 [F736F3D3B39E48818FBA35913AC7CB27]
2016-01-14 17:16:34,852 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791794527_0001_1, queueSize=0
2016-01-14 17:16:34,853 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:34,853 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791794527_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791794527_0001_1, startTime=1452791794691, finishTime=1452791794850, timeTaken=159, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=571, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=16, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1516240896, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905583589597, Process_Duration=10, Process_End_Time=2905583589607, Read_Duration=2, Tuples_Read=40, Tuples_Written=40, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 17:16:34,853 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791794527_0001_1 finished with state: SUCCEEDED
2016-01-14 17:16:34,853 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791794527_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:34,853 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452791794527_0001_00_000002
2016-01-14 17:16:34,853 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791794527_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:16:34 Completed Dag: dag_1452791794527_0001_1
2016-01-14 17:16:34,854 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791794527_0001_1, dagState=SUCCEEDED
2016-01-14 17:16:34,854 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791794527_0001_1_post
2016-01-14 17:16:34,854 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:34,854 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:34,859 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...blyplatform/0_'field'_ALL, applicationId=application_1452791794527_0001
2016-01-14 17:16:34,861 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:34,861 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:34,861 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:34,861 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:34,861 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:34,861 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:34,862 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:34,862 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:34,862 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:34,862 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:34,862 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:34,863 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:34,863 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:34,869 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E53FD299CF414C82B836115DDC358FC6/C7FD7154468C48C080D9E1A1829274B4/.tez/application_1452791794527_0001
2016-01-14 17:16:34,870 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.359
2016-01-14 17:16:34,945 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_'field'_ALL
2016-01-14 17:16:34,966 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:34,968 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:34,994 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.004
2016-01-14 17:16:34,995 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.005
2016-01-14 17:16:34,995 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 17:16:34,996 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 17:16:34,996 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_'field'_REPLACE
2016-01-14 17:16:35,018 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:35,018 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:35,048 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.009
2016-01-14 17:16:35,056 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.017
2016-01-14 17:16:35,056 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:35,057 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:35,057 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:35,070 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:35,077 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_SWAP
2016-01-14 17:16:35,130 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:35,130 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:35,130 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_SWAP"]
2016-01-14 17:16:35,131 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:35,131 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:35,131 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:35,132 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...lyplatform/0_'field'_SWAP
2016-01-14 17:16:35,132 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:35,143 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:35,148 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:35,148 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:35,148 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:35,154 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 17:16:35,154 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 17:16:35,158 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/6925F76E96A1489D92D60DD2AD2D80CE/ABBE7CDD2E2F4AC38B66E5C3017660F7/.tez/application_1452791795148_0001 doesn't exist and is created
2016-01-14 17:16:35,176 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:35,176 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:35,177 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/6925F76E96A1489D92D60DD2AD2D80CE/ABBE7CDD2E2F4AC38B66E5C3017660F7/.tez/application_1452791795148_0001
2016-01-14 17:16:35,180 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791795148_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:35,190 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:35,191 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:35,193 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:35,193 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:35,193 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/6925F76E96A1489D92D60DD2AD2D80CE/ABBE7CDD2E2F4AC38B66E5C3017660F7/.tez/application_1452791795148_0001/localmode-log-dir
2016-01-14 17:16:35,193 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/6925F76E96A1489D92D60DD2AD2D80CE/ABBE7CDD2E2F4AC38B66E5C3017660F7/.tez/application_1452791795148_0001/localmode-log-dir/history.txt.appattempt_1452791795148_0001_000000, maxErrors=10
2016-01-14 17:16:35,194 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:35,195 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791795148_0001_000000, appSubmitTime=1452791795180, launchTime=1452791795180
2016-01-14 17:16:35,200 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:38124
2016-01-14 17:16:35,200 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:35,201 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:35,206 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:35,207 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:35,207 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:35,209 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791795148_0001_000000, startTime=1452791795209
2016-01-14 17:16:35,209 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:35,209 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:35,276 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:35,277 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:35,277 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...lyplatform/0_'field'_SWAP, applicationId=application_1452791795148_0001, dagName=[6925F76E96A1489D92D60DD2AD2D80CE/ABBE7CDD2E2F4AC38B66E5C3017660F7] every/(1/1) ...lyplatform/0_'field'_SWAP
2016-01-14 17:16:35,296 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [6925F76E96A1489D92D60DD2AD2D80CE/ABBE7CDD2E2F4AC38B66E5C3017660F7] every/(1/1) ...lyplatform/0_'field'_SWAP
2016-01-14 17:16:35,297 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791795148_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/6925F76E96A1489D92D60DD2AD2D80CE/ABBE7CDD2E2F4AC38B66E5C3017660F7/.tez/application_1452791795148_0001/localmode-log-dir/dag_1452791795148_0001_1.dot
2016-01-14 17:16:35,297 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/6925F76E96A1489D92D60DD2AD2D80CE/ABBE7CDD2E2F4AC38B66E5C3017660F7/.tez/application_1452791795148_0001/localmode-log-dir/dag_1452791795148_0001_1-tez-dag.pb.txt
2016-01-14 17:16:35,303 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791795148_0001_1
2016-01-14 17:16:35,303 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:35,303 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [6925F76E96A1489D92D60DD2AD2D80CE/ABBE7CDD2E2F4AC38B66E5C3017660F7] every/(1/1) ...lyplatform/0_'field'_SWAP
2016-01-14 17:16:35 Running Dag: dag_1452791795148_0001_1
2016-01-14 17:16:35,316 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791795148_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791795148_0001_1, submitTime=1452791795296
2016-01-14 17:16:35,316 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:35,317 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex EDDC39A6A0EA48AC86029DCC73D1139A
2016-01-14 17:16:35,317 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:35,317 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791795148_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791795148_0001_1, initTime=1452791795317
2016-01-14 17:16:35,318 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791795148_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:35,317 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:35,318 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791795148_0001_1][Event:DAG_STARTED]: dagID=dag_1452791795148_0001_1, startTime=1452791795318
2016-01-14 17:16:35,318 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:35,318 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791795148_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:35,319 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: B98BD3606070402C83177764A2FDDB0B : {C222230215494AB4B2AF7F03E7A7406E={InputName=C222230215494AB4B2AF7F03E7A7406E}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:35,319 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: C222230215494AB4B2AF7F03E7A7406E, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:35,319 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791795148_0001_1_00 [B98BD3606070402C83177764A2FDDB0B]
2016-01-14 17:16:35,319 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791795148_0001_1_00 [B98BD3606070402C83177764A2FDDB0B]
2016-01-14 17:16:35,319 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791795148_0001_1_00 [B98BD3606070402C83177764A2FDDB0B]
2016-01-14 17:16:35,319 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791795148_0001_1_00 [B98BD3606070402C83177764A2FDDB0B]. Starting root input initializers: 1
2016-01-14 17:16:35,318 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:35,320 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791795148_0001_1_00 [B98BD3606070402C83177764A2FDDB0B] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:35,320 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: C222230215494AB4B2AF7F03E7A7406E on vertex vertex_1452791795148_0001_1_00 [B98BD3606070402C83177764A2FDDB0B]
2016-01-14 17:16:35,320 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791795148_0001_1_01 [EDDC39A6A0EA48AC86029DCC73D1139A]
2016-01-14 17:16:35,323 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:35,323 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791795148_0001_1_01 [EDDC39A6A0EA48AC86029DCC73D1139A]
2016-01-14 17:16:35,323 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791795148_0001_1_01 [EDDC39A6A0EA48AC86029DCC73D1139A]
2016-01-14 17:16:35,323 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791795148_0001_1_01 [EDDC39A6A0EA48AC86029DCC73D1139A]
2016-01-14 17:16:35,323 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=CD42FDB659C04CE48C121E0C7366F3EA, vertexId=vertex_1452791795148_0001_1_01 [EDDC39A6A0EA48AC86029DCC73D1139A], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:35,323 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=CD42FDB659C04CE48C121E0C7366F3EA, vertexId=vertex_1452791795148_0001_1_01 [EDDC39A6A0EA48AC86029DCC73D1139A]
2016-01-14 17:16:35,324 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:35,324 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input C222230215494AB4B2AF7F03E7A7406E asking for 1 tasks. Headroom: 723 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:35,324 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:35,324 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:35,325 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=CD42FDB659C04CE48C121E0C7366F3EA, vertexName=EDDC39A6A0EA48AC86029DCC73D1139A, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:35,325 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=CD42FDB659C04CE48C121E0C7366F3EA, vertexId=vertex_1452791795148_0001_1_01 [EDDC39A6A0EA48AC86029DCC73D1139A]
2016-01-14 17:16:35,326 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:35,327 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...lyplatform/0_'field'_SWAP, applicationId=application_1452791795148_0001, dagName=[6925F76E96A1489D92D60DD2AD2D80CE/ABBE7CDD2E2F4AC38B66E5C3017660F7] every/(1/1) ...lyplatform/0_'field'_SWAP
2016-01-14 17:16:35,326 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: C222230215494AB4B2AF7F03E7A7406E on vertex vertex_1452791795148_0001_1_00 [B98BD3606070402C83177764A2FDDB0B]
2016-01-14 17:16:35,331 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:35,331 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791795148_0001_1][Event:VERTEX_INITIALIZED]: vertexName=EDDC39A6A0EA48AC86029DCC73D1139A, vertexId=vertex_1452791795148_0001_1_01, initRequestedTime=1452791795320, initedTime=1452791795326, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:35,331 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791795148_0001_1_01 [EDDC39A6A0EA48AC86029DCC73D1139A] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:35,331 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791795148_0001_1_00 [B98BD3606070402C83177764A2FDDB0B] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:35,333 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {C222230215494AB4B2AF7F03E7A7406E=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:35,333 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791795148_0001_1_00 [B98BD3606070402C83177764A2FDDB0B] parallelism set to 1
2016-01-14 17:16:35,333 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791795148_0001_1_00 [B98BD3606070402C83177764A2FDDB0B] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:35,333 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791795148_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791795148_0001_1_00, eventCount=1
2016-01-14 17:16:35,336 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:35,336 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791795148_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B98BD3606070402C83177764A2FDDB0B, vertexId=vertex_1452791795148_0001_1_00, initRequestedTime=1452791795318, initedTime=1452791795335, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:35,336 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791795148_0001_1_00 [B98BD3606070402C83177764A2FDDB0B] with distanceFromRoot: 0
2016-01-14 17:16:35,336 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791795148_0001_1_00 [B98BD3606070402C83177764A2FDDB0B] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:35,337 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in B98BD3606070402C83177764A2FDDB0B
2016-01-14 17:16:35,337 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791795148_0001_1_00 [B98BD3606070402C83177764A2FDDB0B]
2016-01-14 17:16:35,337 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:35,337 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791795148_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791795148_0001_1_00, startRequestedTime=1452791795337, startedTime=1452791795337
2016-01-14 17:16:35,338 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791795148_0001_1_00 [B98BD3606070402C83177764A2FDDB0B] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:35,338 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:35,338 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791795148_0001_1][Event:TASK_STARTED]: vertexName=B98BD3606070402C83177764A2FDDB0B, taskId=task_1452791795148_0001_1_00_000000, scheduledTime=1452791795338, launchTime=1452791795338
2016-01-14 17:16:35,338 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791795148_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:35,339 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791795148_0001_1_00 for vertex: vertex_1452791795148_0001_1_01 [EDDC39A6A0EA48AC86029DCC73D1139A] numStartedSources: 1 numSources: 1
2016-01-14 17:16:35,339 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791795148_0001_1_01 [EDDC39A6A0EA48AC86029DCC73D1139A] with distanceFromRoot: 1
2016-01-14 17:16:35,339 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791795148_0001_1_00 [B98BD3606070402C83177764A2FDDB0B]
2016-01-14 17:16:35,339 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: B98BD3606070402C83177764A2FDDB0B in vertex: EDDC39A6A0EA48AC86029DCC73D1139A
2016-01-14 17:16:35,340 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: EDDC39A6A0EA48AC86029DCC73D1139A with 1 source tasks and 1 pending tasks
2016-01-14 17:16:35,340 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:35,340 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791795148_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791795148_0001_1_01, startRequestedTime=1452791795339, startedTime=1452791795339
2016-01-14 17:16:35,340 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791795148_0001_1_01 [EDDC39A6A0EA48AC86029DCC73D1139A] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:35,341 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [6925F76E96A1489D92D60DD2AD2D80CE/ABBE7CDD2E2F4AC38B66E5C3017660F7] every/(1/1) ...lyplatform/0_'field'_SWAP, VertexName: B98BD3606070402C83177764A2FDDB0B, VertexParallelism: 1, TaskAttemptID:attempt_1452791795148_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C222230215494AB4B2AF7F03E7A7406E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=EDDC39A6A0EA48AC86029DCC73D1139A, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:35,341 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791795148_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:35,341 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:35,342 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:35,342 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:35,342 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:35,343 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791795148_0001_00_000001
2016-01-14 17:16:35,343 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791795148_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:35,344 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791795148_0001_00_000001
2016-01-14 17:16:35,344 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:35,345 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791795148_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:35,345 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791795148_0001, with dag id: dag_1452791795148_0001_1
2016-01-14 17:16:35,345 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791795148_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791795148_0001_00_000001, launchTime=1452791795345
2016-01-14 17:16:35,345 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:35,359 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791795148_0001_00_000001
2016-01-14 17:16:35,362 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791795148_0001_1_00_000000_0] to container: [container_1452791795148_0001_00_000001]
2016-01-14 17:16:35,363 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791795148_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:35,364 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791795148_0001_1_00_000000_0] started. Is using containerId: [container_1452791795148_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:35,364 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:35,364 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791795148_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B98BD3606070402C83177764A2FDDB0B, taskAttemptId=attempt_1452791795148_0001_1_00_000000_0, startTime=1452791795363, containerId=container_1452791795148_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791795148_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:35,364 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791795148_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:35,364 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791795148_0001_00_000001 given task: attempt_1452791795148_0001_1_00_000000_0
2016-01-14 17:16:35,364 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791795148_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:35,364 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791795148_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791795148_0001_1_00_000000_0
2016-01-14 17:16:35 Starting to run new task attempt: attempt_1452791795148_0001_1_00_000000_0
2016-01-14 17:16:35,365 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791795148_0001_1_00_000000_0
2016-01-14 17:16:35,365 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:35,365 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:35,365 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:35,365 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:35,365 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:35,365 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:35,365 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [6925F76E96A1489D92D60DD2AD2D80CE/ABBE7CDD2E2F4AC38B66E5C3017660F7] every/(1/1) ...lyplatform/0_'field'_SWAP, VertexName: B98BD3606070402C83177764A2FDDB0B, VertexParallelism: 1, TaskAttemptID:attempt_1452791795148_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C222230215494AB4B2AF7F03E7A7406E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=EDDC39A6A0EA48AC86029DCC73D1139A, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:35,366 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=755499008
2016-01-14 17:16:35,366 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791795148_0001_1_00_000000_0
2016-01-14 17:16:35,366 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:35,366 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:35,367 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791795148_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:35,367 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C222230215494AB4B2AF7F03E7A7406E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:35,368 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=EDDC39A6A0EA48AC86029DCC73D1139A, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:35,367 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:35,368 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:35,368 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C222230215494AB4B2AF7F03E7A7406E
2016-01-14 17:16:35,369 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: C222230215494AB4B2AF7F03E7A7406E
2016-01-14 17:16:35,369 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:35,369 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: EDDC39A6A0EA48AC86029DCC73D1139A
2016-01-14 17:16:35,371 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:35,371 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:35,371 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: C222230215494AB4B2AF7F03E7A7406E
2016-01-14 17:16:35,371 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C222230215494AB4B2AF7F03E7A7406E
2016-01-14 17:16:35,371 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:35,372 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: EDDC39A6A0EA48AC86029DCC73D1139A
2016-01-14 17:16:35,372 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: EDDC39A6A0EA48AC86029DCC73D1139A
2016-01-14 17:16:35,375 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B98BD3606070402C83177764A2FDDB0B, ordinal: 0
2016-01-14 17:16:35,375 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B98BD3606070402C83177764A2FDDB0B, mem on start (mb), free: 477, total: 720, max: 720
2016-01-14 17:16:35,375 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:35,375 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:35,375 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:35,375 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:35,375 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:35,376 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:35,376 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:35,376 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:35,376 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:35,377 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 755499008, TotalAvailable: 506184335, TotalRequested/TotalJVMHeap:0.14
2016-01-14 17:16:35,377 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C222230215494AB4B2AF7F03E7A7406E, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:35,377 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, EDDC39A6A0EA48AC86029DCC73D1139A, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:35,377 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:35,377 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C222230215494AB4B2AF7F03E7A7406E being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:35,378 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:35,378 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:35,377 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C222230215494AB4B2AF7F03E7A7406E
2016-01-14 17:16:35,379 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:35,379 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C222230215494AB4B2AF7F03E7A7406E
2016-01-14 17:16:35,379 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:35,380 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791795148_0001_1_00_000000_0
2016-01-14 17:16:35,381 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:35,382 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:35,383 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: C222230215494AB4B2AF7F03E7A7406E
2016-01-14 17:16:35,383 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 83C762363E7E4F578427D4866A30B237
2016-01-14 17:16:35,383 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 83C762363E7E4F578427D4866A30B237
2016-01-14 17:16:35,384 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@11fc864d
2016-01-14 17:16:35,384 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@59d53274; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@3be38bbf; comparator=cascading.tuple.hadoop.util.TupleComparator@57ebc750; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:35,384 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:35,384 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:35,385 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:35,385 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@11fc864d
2016-01-14 17:16:35,385 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:35,385 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:35,398 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:35,398 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:35,398 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:35,398 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:35,398 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:35,399 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:35,399 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:35,399 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B98BD3606070402C83177764A2FDDB0B, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:35,401 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791795148_0001_1_00_000000_0
2016-01-14 17:16:35,401 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B98BD3606070402C83177764A2FDDB0B, mem on close (mb), free: 376, total: 720, max: 720
2016-01-14 17:16:35,402 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:35,402 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:35,402 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:35,402 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:35,404 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:35 Completed running task attempt: attempt_1452791795148_0001_1_00_000000_0
2016-01-14 17:16:35,407 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791795148_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:35,407 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791795148_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791795148_0001_1_00, eventCount=1
2016-01-14 17:16:35,407 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791795148_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B98BD3606070402C83177764A2FDDB0B, taskAttemptId=attempt_1452791795148_0001_1_00_000000_0, startTime=1452791795363, finishTime=1452791795407, timeTaken=44, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=755499008, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791795399, Process_Duration=2, Process_End_Time=1452791795401, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:35,408 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791795148_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:35,408 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791795148_0001_1_00_000000_0
2016-01-14 17:16:35,408 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:35,408 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:35,408 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791795148_0001_1][Event:TASK_FINISHED]: vertexName=B98BD3606070402C83177764A2FDDB0B, taskId=task_1452791795148_0001_1_00_000000, startTime=1452791795363, finishTime=1452791795408, timeTaken=45, status=SUCCEEDED, successfulAttemptID=attempt_1452791795148_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=755499008, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791795399, Process_Duration=2, Process_End_Time=1452791795401, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:35,409 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:35,409 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791795148_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:35,409 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791795148_0001_1_00 [B98BD3606070402C83177764A2FDDB0B] : 1
2016-01-14 17:16:35,409 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791795148_0001_1_00 [B98BD3606070402C83177764A2FDDB0B]
2016-01-14 17:16:35,410 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:35,410 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791795148_0001_1][Event:VERTEX_FINISHED]: vertexName=B98BD3606070402C83177764A2FDDB0B, vertexId=vertex_1452791795148_0001_1_00, initRequestedTime=1452791795318, initedTime=1452791795335, startRequestedTime=1452791795337, startedTime=1452791795337, finishTime=1452791795409, timeTaken=72, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=755499008, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791795399, Process_Duration=2, Process_End_Time=1452791795401, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791795363, firstTasksToStart=[ task_1452791795148_0001_1_00_000000 ], lastTaskFinishTime=1452791795407, lastTasksToFinish=[ task_1452791795148_0001_1_00_000000 ], minTaskDuration=44, maxTaskDuration=44, avgTaskDuration=44.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791795148_0001_1_00_000000 ], longestDurationTasks=[ task_1452791795148_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:35,410 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791795148_0001_1_00 [B98BD3606070402C83177764A2FDDB0B] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:35,410 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791795148_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:35,410 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791795148_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791795148_0001_00_000001, stoppedTime=1452791795410, exitStatus=0
2016-01-14 17:16:35,410 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:35,411 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791795148_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:35,411 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791795148_0001_1_01 [EDDC39A6A0EA48AC86029DCC73D1139A] attempt: attempt_1452791795148_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:35,411 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: EDDC39A6A0EA48AC86029DCC73D1139A
2016-01-14 17:16:35,411 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791795148_0001_1_01 [EDDC39A6A0EA48AC86029DCC73D1139A]
2016-01-14 17:16:35,411 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791795148_0001_1_00 [B98BD3606070402C83177764A2FDDB0B] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:35,411 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791795148_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:35,411 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791795148_0001_1][Event:TASK_STARTED]: vertexName=EDDC39A6A0EA48AC86029DCC73D1139A, taskId=task_1452791795148_0001_1_01_000000, scheduledTime=1452791795411, launchTime=1452791795411
2016-01-14 17:16:35,411 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:35,412 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791795148_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:35,412 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791795148_0001_1_01 [EDDC39A6A0EA48AC86029DCC73D1139A]
2016-01-14 17:16:35,412 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791795148_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:35,412 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [6925F76E96A1489D92D60DD2AD2D80CE/ABBE7CDD2E2F4AC38B66E5C3017660F7] every/(1/1) ...lyplatform/0_'field'_SWAP, VertexName: EDDC39A6A0EA48AC86029DCC73D1139A, VertexParallelism: 1, TaskAttemptID:attempt_1452791795148_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B98BD3606070402C83177764A2FDDB0B, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=CD42FDB659C04CE48C121E0C7366F3EA, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:35,412 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791795148_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:35,412 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:35,412 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=755499008, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791795399, Process_Duration=2, Process_End_Time=1452791795401, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:35,413 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791795148_0001_00_000002
2016-01-14 17:16:35,413 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791795148_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:35,413 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:35,413 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:35,413 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791795148_0001_00_000001
2016-01-14 17:16:35,413 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791795148_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:35,413 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791795148_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:35,414 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791795148_0001_00_000001, Exiting
2016-01-14 17:16:35,414 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791795148_0001_00_000001
2016-01-14 17:16:35,414 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791795148_0001_00_000001
2016-01-14 17:16:35,414 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791795148_0001_00_000002
2016-01-14 17:16:35,415 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 17:16:35,415 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791795148_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:35,415 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791795148_0001_00_000001 completed successfully
2016-01-14 17:16:35,416 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791795148_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:35,417 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791795148_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:35,417 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:35,418 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791795148_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791795148_0001_00_000002, launchTime=1452791795417
2016-01-14 17:16:35,418 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791795148_0001_00_000002
2016-01-14 17:16:35,419 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791795148_0001_1_01_000000_0] to container: [container_1452791795148_0001_00_000002]
2016-01-14 17:16:35,419 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791795148_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:35,419 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791795148_0001_00_000002 given task: attempt_1452791795148_0001_1_01_000000_0
2016-01-14 17:16:35,419 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791795148_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791795148_0001_1_01_000000_0
2016-01-14 17:16:35 Starting to run new task attempt: attempt_1452791795148_0001_1_01_000000_0
2016-01-14 17:16:35,419 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791795148_0001_1_01_000000_0] started. Is using containerId: [container_1452791795148_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:35,419 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791795148_0001_1_01_000000_0
2016-01-14 17:16:35,420 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:35,420 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:35,420 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791795148_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=EDDC39A6A0EA48AC86029DCC73D1139A, taskAttemptId=attempt_1452791795148_0001_1_01_000000_0, startTime=1452791795419, containerId=container_1452791795148_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791795148_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:35,420 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791795148_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:35,420 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791795148_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:35,420 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:35,421 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:35,421 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:35,421 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:35,421 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:35,422 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [6925F76E96A1489D92D60DD2AD2D80CE/ABBE7CDD2E2F4AC38B66E5C3017660F7] every/(1/1) ...lyplatform/0_'field'_SWAP, VertexName: EDDC39A6A0EA48AC86029DCC73D1139A, VertexParallelism: 1, TaskAttemptID:attempt_1452791795148_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B98BD3606070402C83177764A2FDDB0B, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=CD42FDB659C04CE48C121E0C7366F3EA, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:35,422 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=755499008
2016-01-14 17:16:35,422 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791795148_0001_1_01_000000_0
2016-01-14 17:16:35,423 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:35,423 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:35,423 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B98BD3606070402C83177764A2FDDB0B, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:35,423 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791795148_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:35,424 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:35,424 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:35,424 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B98BD3606070402C83177764A2FDDB0B
2016-01-14 17:16:35,426 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 679949120, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:35,426 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:35,426 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 679949120, type: INPUT, componentVertexName: B98BD3606070402C83177764A2FDDB0B
2016-01-14 17:16:35,426 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B98BD3606070402C83177764A2FDDB0B
2016-01-14 17:16:35,426 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:35,431 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=CD42FDB659C04CE48C121E0C7366F3EA, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:35,432 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: EDDC39A6A0EA48AC86029DCC73D1139A, ordinal: 1
2016-01-14 17:16:35,433 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EDDC39A6A0EA48AC86029DCC73D1139A, mem on start (mb), free: 371, total: 720, max: 720
2016-01-14 17:16:35,433 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:35,433 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:35,433 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:35,434 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:35,434 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: CD42FDB659C04CE48C121E0C7366F3EA
2016-01-14 17:16:35,434 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:35,434 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: CD42FDB659C04CE48C121E0C7366F3EA
2016-01-14 17:16:35,444 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:35,445 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: CD42FDB659C04CE48C121E0C7366F3EA
2016-01-14 17:16:35,445 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:35,445 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:35,445 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:35,445 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:35,445 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:35,446 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:35,446 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 679949120, TotalRequestedScaled: 6.276453415384616E8, TotalJVMHeap: 755499008, TotalAvailable: 506184335, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:35,446 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B98BD3606070402C83177764A2FDDB0B, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=679949120, allocated=506184335
2016-01-14 17:16:35,446 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, CD42FDB659C04CE48C121E0C7366F3EA, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:35,446 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:35,447 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B98BD3606070402C83177764A2FDDB0B being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:35,447 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B98BD3606070402C83177764A2FDDB0B
2016-01-14 17:16:35,447 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:35,448 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:35,448 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:35,449 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:35,449 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: B98BD3606070402C83177764A2FDDB0B with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:35,449 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=679949120, postMergeMem=0, RuntimeTotalAvailable=506184335. Updated to: ShuffleMem=506184335, postMergeMem: 0
2016-01-14 17:16:35,449 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=506184335, maxSingleShuffleLimit=126546080, mergeThreshold=455565888, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:35,449 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:35,451 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B98BD3606070402C83177764A2FDDB0B
2016-01-14 17:16:35,451 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:35,451 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791795148_0001_1_01_000000_0
2016-01-14 17:16:35,454 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791795148_0001_1_00_000000_0_10141, runDuration: 36269, ]
2016-01-14 17:16:35,456 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791795148_0001_1_00_000000_0_10141] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791795455, TimeTaken=0, Rate=0.00 MB/s
2016-01-14 17:16:35,456 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:35,456 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : B98BD3606070402C83177764A2FDDB0B
2016-01-14 17:16:35,456 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:35,456 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 83C762363E7E4F578427D4866A30B237
2016-01-14 17:16:35,457 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_SWAP"], id: CD42FDB659C04CE48C121E0C7366F3EA
2016-01-14 17:16:35,457 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_SWAP"]
2016-01-14 17:16:35,457 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 83C762363E7E4F578427D4866A30B237, for 1 inputs
2016-01-14 17:16:35,456 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [B98BD3606070402C83177764A2FDDB0B] #24 in 1ms
2016-01-14 17:16:35,457 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [B98BD3606070402C83177764A2FDDB0B] #24, status:false, isInterrupted:false
2016-01-14 17:16:35,458 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:35,458 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:35,458 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:35,458 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:35,458 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:35,458 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : B98BD3606070402C83177764A2FDDB0B
2016-01-14 17:16:35,458 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:35,458 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: EDDC39A6A0EA48AC86029DCC73D1139A, all 1 inputs ready in: 00:00:00.001
2016-01-14 17:16:35,459 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@2d44c0b1; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@7cc6bbae
2016-01-14 17:16:35,463 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:35,463 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:35,463 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791795148_0001_1_01_000000_0
2016-01-14 17:16:35,464 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791795148_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:35,464 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791795148_0001_1_01_000000_0
2016-01-14 17:16:35,464 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EDDC39A6A0EA48AC86029DCC73D1139A, mem on close (mb), free: 367, total: 720, max: 720
2016-01-14 17:16:35,465 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: B98BD3606070402C83177764A2FDDB0B
2016-01-14 17:16:35,465 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:35,465 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:35,465 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:35,465 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:35,466 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:35 Completed running task attempt: attempt_1452791795148_0001_1_01_000000_0
2016-01-14 17:16:35,466 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791795148_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:35,467 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=183, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=755499008, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791795458, Process_Duration=6, Process_End_Time=1452791795464, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 17:16:35,467 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:35,468 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:35,467 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791795148_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=EDDC39A6A0EA48AC86029DCC73D1139A, taskAttemptId=attempt_1452791795148_0001_1_01_000000_0, startTime=1452791795419, finishTime=1452791795467, timeTaken=48, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=183, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=755499008, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791795458, Process_Duration=6, Process_End_Time=1452791795464, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:35,468 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791795148_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:35,468 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:35,468 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:35,468 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791795148_0001_1_01_000000_0
2016-01-14 17:16:35,469 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:35,469 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791795148_0001_1][Event:TASK_FINISHED]: vertexName=EDDC39A6A0EA48AC86029DCC73D1139A, taskId=task_1452791795148_0001_1_01_000000, startTime=1452791795419, finishTime=1452791795469, timeTaken=50, status=SUCCEEDED, successfulAttemptID=attempt_1452791795148_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=183, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=755499008, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791795458, Process_Duration=6, Process_End_Time=1452791795464, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:35,469 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791795148_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:35,469 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791795148_0001_1_01 [EDDC39A6A0EA48AC86029DCC73D1139A] : 1
2016-01-14 17:16:35,469 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791795148_0001_00_000002
2016-01-14 17:16:35,470 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791795148_0001_1_01 [EDDC39A6A0EA48AC86029DCC73D1139A]
2016-01-14 17:16:35,471 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:35,471 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791795148_0001_1][Event:VERTEX_FINISHED]: vertexName=EDDC39A6A0EA48AC86029DCC73D1139A, vertexId=vertex_1452791795148_0001_1_01, initRequestedTime=1452791795320, initedTime=1452791795326, startRequestedTime=1452791795339, startedTime=1452791795339, finishTime=1452791795470, timeTaken=131, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=183, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=755499008, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791795458, Process_Duration=6, Process_End_Time=1452791795464, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452791795419, firstTasksToStart=[ task_1452791795148_0001_1_01_000000 ], lastTaskFinishTime=1452791795467, lastTasksToFinish=[ task_1452791795148_0001_1_01_000000 ], minTaskDuration=48, maxTaskDuration=48, avgTaskDuration=48.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791795148_0001_1_01_000000 ], longestDurationTasks=[ task_1452791795148_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:35,471 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791795148_0001_1_01 [EDDC39A6A0EA48AC86029DCC73D1139A] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:35,471 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791795148_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:35,471 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:35,471 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791795148_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791795148_0001_00_000002, stoppedTime=1452791795471, exitStatus=0
2016-01-14 17:16:35,472 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791795148_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:35,472 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791795148_0001_1_01 [EDDC39A6A0EA48AC86029DCC73D1139A] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:35,472 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791795148_0001_1
2016-01-14 17:16:35,473 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791795148_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452791795148_0001_1
2016-01-14 17:16:35,473 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: CD42FDB659C04CE48C121E0C7366F3EA for vertex: vertex_1452791795148_0001_1_01 [EDDC39A6A0EA48AC86029DCC73D1139A]
2016-01-14 17:16:35,480 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791795148_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 17:16:35,481 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791795148_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:35,481 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791795148_0001_00_000002, Exiting
2016-01-14 17:16:35,481 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791795148_0001_00_000002
2016-01-14 17:16:35,481 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791795148_0001_00_000002
2016-01-14 17:16:35,481 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791795148_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:35,481 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791795148_0001_00_000002 completed successfully
2016-01-14 17:16:35,482 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452791795148_0001_1_00 [B98BD3606070402C83177764A2FDDB0B]
2016-01-14 17:16:35,485 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791795148_0001_1, queueSize=0
2016-01-14 17:16:35,485 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:35,485 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791795148_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791795148_0001_1, startTime=1452791795318, finishTime=1452791795482, timeTaken=164, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=516, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1510998016, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905583590857, Process_Duration=8, Process_End_Time=2905583590865, Read_Duration=2, Tuples_Read=40, Tuples_Written=40, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 17:16:35,485 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791795148_0001_1 finished with state: SUCCEEDED
2016-01-14 17:16:35,485 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791795148_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:35,485 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791795148_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:16:35 Completed Dag: dag_1452791795148_0001_1
2016-01-14 17:16:35,486 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791795148_0001_1, dagState=SUCCEEDED
2016-01-14 17:16:35,486 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791795148_0001_1_post
2016-01-14 17:16:35,486 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:35,486 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:35,486 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452791795148_0001_00_000002
2016-01-14 17:16:35,492 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...lyplatform/0_'field'_SWAP, applicationId=application_1452791795148_0001
2016-01-14 17:16:35,494 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:35,494 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:35,494 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:35,494 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:35,494 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:35,495 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:35,495 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:35,495 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:35,495 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:35,495 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:35,495 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:35,496 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:35,496 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:35,500 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/6925F76E96A1489D92D60DD2AD2D80CE/ABBE7CDD2E2F4AC38B66E5C3017660F7/.tez/application_1452791795148_0001
2016-01-14 17:16:35,501 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.371
2016-01-14 17:16:35,565 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_'field'_SWAP
2016-01-14 17:16:35,587 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:35,587 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:35,620 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-01-14 17:16:35,627 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.016
2016-01-14 17:16:35,627 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:35,627 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:35,627 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:35,640 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:35,645 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_-1
2016-01-14 17:16:35,687 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:35,687 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:35,688 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_-1"]
2016-01-14 17:16:35,688 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:35,688 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:35,688 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:35,689 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...mblyplatform/0_UNKNOWN_-1
2016-01-14 17:16:35,689 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:35,693 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 17:16:35,693 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 17:16:35,701 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:35,705 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:35,706 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:35,706 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:35,715 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/9B13F80AEFFB491CA058B9DE9CC0912D/FBFE8631A6944DB992E88A9EE5898757/.tez/application_1452791795705_0001 doesn't exist and is created
2016-01-14 17:16:35,730 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:35,730 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:35,730 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/9B13F80AEFFB491CA058B9DE9CC0912D/FBFE8631A6944DB992E88A9EE5898757/.tez/application_1452791795705_0001
2016-01-14 17:16:35,733 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791795705_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:35,743 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:35,743 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:35,744 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:35,745 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:35,746 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/9B13F80AEFFB491CA058B9DE9CC0912D/FBFE8631A6944DB992E88A9EE5898757/.tez/application_1452791795705_0001/localmode-log-dir
2016-01-14 17:16:35,746 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/9B13F80AEFFB491CA058B9DE9CC0912D/FBFE8631A6944DB992E88A9EE5898757/.tez/application_1452791795705_0001/localmode-log-dir/history.txt.appattempt_1452791795705_0001_000000, maxErrors=10
2016-01-14 17:16:35,747 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:35,747 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791795705_0001_000000, appSubmitTime=1452791795733, launchTime=1452791795733
2016-01-14 17:16:35,750 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:35,753 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:35,754 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:35,754 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:35,757 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:43919
2016-01-14 17:16:35,760 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:35,761 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791795705_0001_000000, startTime=1452791795761
2016-01-14 17:16:35,761 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:35,761 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:35,830 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:35,830 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:35,830 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...mblyplatform/0_UNKNOWN_-1, applicationId=application_1452791795705_0001, dagName=[9B13F80AEFFB491CA058B9DE9CC0912D/FBFE8631A6944DB992E88A9EE5898757] every/(1/1) ...mblyplatform/0_UNKNOWN_-1
2016-01-14 17:16:35,846 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [9B13F80AEFFB491CA058B9DE9CC0912D/FBFE8631A6944DB992E88A9EE5898757] every/(1/1) ...mblyplatform/0_UNKNOWN_-1
2016-01-14 17:16:35,847 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791795705_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/9B13F80AEFFB491CA058B9DE9CC0912D/FBFE8631A6944DB992E88A9EE5898757/.tez/application_1452791795705_0001/localmode-log-dir/dag_1452791795705_0001_1.dot
2016-01-14 17:16:35,847 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/9B13F80AEFFB491CA058B9DE9CC0912D/FBFE8631A6944DB992E88A9EE5898757/.tez/application_1452791795705_0001/localmode-log-dir/dag_1452791795705_0001_1-tez-dag.pb.txt
2016-01-14 17:16:35,854 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791795705_0001_1
2016-01-14 17:16:35,854 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:35,854 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [9B13F80AEFFB491CA058B9DE9CC0912D/FBFE8631A6944DB992E88A9EE5898757] every/(1/1) ...mblyplatform/0_UNKNOWN_-1
2016-01-14 17:16:35 Running Dag: dag_1452791795705_0001_1
2016-01-14 17:16:35,866 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791795705_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791795705_0001_1, submitTime=1452791795846
2016-01-14 17:16:35,866 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:35,866 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex A79979D16F6A47C280295E4BB34D1669
2016-01-14 17:16:35,867 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:35,867 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791795705_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791795705_0001_1, initTime=1452791795866
2016-01-14 17:16:35,867 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:35,867 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791795705_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:35,868 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791795705_0001_1][Event:DAG_STARTED]: dagID=dag_1452791795705_0001_1, startTime=1452791795868
2016-01-14 17:16:35,868 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:35,868 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:35,868 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791795705_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:35,868 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C2752992A89549B6B8A8C0009345CD6F : {D2EA2B42DDDD4171B51B5F281EC37A43={InputName=D2EA2B42DDDD4171B51B5F281EC37A43}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:35,868 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: D2EA2B42DDDD4171B51B5F281EC37A43, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:35,868 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791795705_0001_1_00 [C2752992A89549B6B8A8C0009345CD6F]
2016-01-14 17:16:35,868 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791795705_0001_1_00 [C2752992A89549B6B8A8C0009345CD6F]
2016-01-14 17:16:35,868 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791795705_0001_1_00 [C2752992A89549B6B8A8C0009345CD6F]
2016-01-14 17:16:35,869 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791795705_0001_1_00 [C2752992A89549B6B8A8C0009345CD6F]. Starting root input initializers: 1
2016-01-14 17:16:35,869 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791795705_0001_1_00 [C2752992A89549B6B8A8C0009345CD6F] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:35,869 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791795705_0001_1_01 [A79979D16F6A47C280295E4BB34D1669]
2016-01-14 17:16:35,870 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...mblyplatform/0_UNKNOWN_-1, applicationId=application_1452791795705_0001, dagName=[9B13F80AEFFB491CA058B9DE9CC0912D/FBFE8631A6944DB992E88A9EE5898757] every/(1/1) ...mblyplatform/0_UNKNOWN_-1
2016-01-14 17:16:35,870 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: D2EA2B42DDDD4171B51B5F281EC37A43 on vertex vertex_1452791795705_0001_1_00 [C2752992A89549B6B8A8C0009345CD6F]
2016-01-14 17:16:35,872 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:35,872 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input D2EA2B42DDDD4171B51B5F281EC37A43 asking for 1 tasks. Headroom: 720 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:35,872 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:35,872 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791795705_0001_1_01 [A79979D16F6A47C280295E4BB34D1669]
2016-01-14 17:16:35,872 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:35,872 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791795705_0001_1_01 [A79979D16F6A47C280295E4BB34D1669]
2016-01-14 17:16:35,873 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:35,873 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791795705_0001_1_01 [A79979D16F6A47C280295E4BB34D1669]
2016-01-14 17:16:35,873 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=FD8C91F533344213B34B61BE65032F43, vertexId=vertex_1452791795705_0001_1_01 [A79979D16F6A47C280295E4BB34D1669], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:35,873 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=FD8C91F533344213B34B61BE65032F43, vertexId=vertex_1452791795705_0001_1_01 [A79979D16F6A47C280295E4BB34D1669]
2016-01-14 17:16:35,875 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=FD8C91F533344213B34B61BE65032F43, vertexName=A79979D16F6A47C280295E4BB34D1669, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:35,875 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=FD8C91F533344213B34B61BE65032F43, vertexId=vertex_1452791795705_0001_1_01 [A79979D16F6A47C280295E4BB34D1669]
2016-01-14 17:16:35,876 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:35,876 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: D2EA2B42DDDD4171B51B5F281EC37A43 on vertex vertex_1452791795705_0001_1_00 [C2752992A89549B6B8A8C0009345CD6F]
2016-01-14 17:16:35,876 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791795705_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A79979D16F6A47C280295E4BB34D1669, vertexId=vertex_1452791795705_0001_1_01, initRequestedTime=1452791795869, initedTime=1452791795876, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:35,876 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:35,876 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791795705_0001_1_01 [A79979D16F6A47C280295E4BB34D1669] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:35,877 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791795705_0001_1_00 [C2752992A89549B6B8A8C0009345CD6F] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:35,877 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {D2EA2B42DDDD4171B51B5F281EC37A43=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:35,877 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791795705_0001_1_00 [C2752992A89549B6B8A8C0009345CD6F] parallelism set to 1
2016-01-14 17:16:35,877 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791795705_0001_1_00 [C2752992A89549B6B8A8C0009345CD6F] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:35,878 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791795705_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791795705_0001_1_00, eventCount=1
2016-01-14 17:16:35,878 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:35,878 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791795705_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C2752992A89549B6B8A8C0009345CD6F, vertexId=vertex_1452791795705_0001_1_00, initRequestedTime=1452791795868, initedTime=1452791795878, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:35,878 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791795705_0001_1_00 [C2752992A89549B6B8A8C0009345CD6F] with distanceFromRoot: 0
2016-01-14 17:16:35,878 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791795705_0001_1_00 [C2752992A89549B6B8A8C0009345CD6F] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:35,879 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C2752992A89549B6B8A8C0009345CD6F
2016-01-14 17:16:35,879 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791795705_0001_1_00 [C2752992A89549B6B8A8C0009345CD6F]
2016-01-14 17:16:35,879 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:35,879 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791795705_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791795705_0001_1_00, startRequestedTime=1452791795879, startedTime=1452791795879
2016-01-14 17:16:35,879 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791795705_0001_1_00 [C2752992A89549B6B8A8C0009345CD6F] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:35,880 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:35,880 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791795705_0001_1][Event:TASK_STARTED]: vertexName=C2752992A89549B6B8A8C0009345CD6F, taskId=task_1452791795705_0001_1_00_000000, scheduledTime=1452791795880, launchTime=1452791795880
2016-01-14 17:16:35,880 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791795705_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:35,881 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791795705_0001_1_00 for vertex: vertex_1452791795705_0001_1_01 [A79979D16F6A47C280295E4BB34D1669] numStartedSources: 1 numSources: 1
2016-01-14 17:16:35,881 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791795705_0001_1_01 [A79979D16F6A47C280295E4BB34D1669] with distanceFromRoot: 1
2016-01-14 17:16:35,881 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791795705_0001_1_00 [C2752992A89549B6B8A8C0009345CD6F]
2016-01-14 17:16:35,881 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: C2752992A89549B6B8A8C0009345CD6F in vertex: A79979D16F6A47C280295E4BB34D1669
2016-01-14 17:16:35,881 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: A79979D16F6A47C280295E4BB34D1669 with 1 source tasks and 1 pending tasks
2016-01-14 17:16:35,882 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:35,882 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791795705_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791795705_0001_1_01, startRequestedTime=1452791795881, startedTime=1452791795881
2016-01-14 17:16:35,882 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791795705_0001_1_01 [A79979D16F6A47C280295E4BB34D1669] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:35,882 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [9B13F80AEFFB491CA058B9DE9CC0912D/FBFE8631A6944DB992E88A9EE5898757] every/(1/1) ...mblyplatform/0_UNKNOWN_-1, VertexName: C2752992A89549B6B8A8C0009345CD6F, VertexParallelism: 1, TaskAttemptID:attempt_1452791795705_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D2EA2B42DDDD4171B51B5F281EC37A43, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A79979D16F6A47C280295E4BB34D1669, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:35,883 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791795705_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:35,883 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:35,883 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:35,884 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:35,884 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:35,884 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791795705_0001_00_000001
2016-01-14 17:16:35,884 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791795705_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:35,886 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791795705_0001_00_000001
2016-01-14 17:16:35,886 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:35,887 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:35,887 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791795705_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791795705_0001_00_000001, launchTime=1452791795887
2016-01-14 17:16:35,888 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791795705_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:35,888 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791795705_0001_00_000001
2016-01-14 17:16:35,889 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791795705_0001_1_00_000000_0] to container: [container_1452791795705_0001_00_000001]
2016-01-14 17:16:35,889 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791795705_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:35,889 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791795705_0001_00_000001 given task: attempt_1452791795705_0001_1_00_000000_0
2016-01-14 17:16:35,889 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791795705_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791795705_0001_1_00_000000_0
2016-01-14 17:16:35,889 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791795705_0001_1_00_000000_0] started. Is using containerId: [container_1452791795705_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:35,890 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:35,890 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791795705_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C2752992A89549B6B8A8C0009345CD6F, taskAttemptId=attempt_1452791795705_0001_1_00_000000_0, startTime=1452791795889, containerId=container_1452791795705_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791795705_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:35,890 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791795705_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:35,891 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791795705_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:35 Starting to run new task attempt: attempt_1452791795705_0001_1_00_000000_0
2016-01-14 17:16:35,891 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791795705_0001_1_00_000000_0
2016-01-14 17:16:35,891 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:35,891 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:35,892 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:35,892 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791795705_0001, with dag id: dag_1452791795705_0001_1
2016-01-14 17:16:35,892 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:35,892 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:35,892 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:35,893 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [9B13F80AEFFB491CA058B9DE9CC0912D/FBFE8631A6944DB992E88A9EE5898757] every/(1/1) ...mblyplatform/0_UNKNOWN_-1, VertexName: C2752992A89549B6B8A8C0009345CD6F, VertexParallelism: 1, TaskAttemptID:attempt_1452791795705_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D2EA2B42DDDD4171B51B5F281EC37A43, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A79979D16F6A47C280295E4BB34D1669, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:35,893 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=755499008
2016-01-14 17:16:35,893 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791795705_0001_1_00_000000_0
2016-01-14 17:16:35,894 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:35,894 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:35,894 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791795705_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:35,895 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=D2EA2B42DDDD4171B51B5F281EC37A43, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:35,896 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:35,896 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: D2EA2B42DDDD4171B51B5F281EC37A43
2016-01-14 17:16:35,896 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: D2EA2B42DDDD4171B51B5F281EC37A43
2016-01-14 17:16:35,897 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:35,898 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A79979D16F6A47C280295E4BB34D1669, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:35,898 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:35,899 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: D2EA2B42DDDD4171B51B5F281EC37A43
2016-01-14 17:16:35,899 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: D2EA2B42DDDD4171B51B5F281EC37A43
2016-01-14 17:16:35,899 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:35,900 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A79979D16F6A47C280295E4BB34D1669
2016-01-14 17:16:35,901 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:35,902 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:35,902 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: A79979D16F6A47C280295E4BB34D1669
2016-01-14 17:16:35,902 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A79979D16F6A47C280295E4BB34D1669
2016-01-14 17:16:35,906 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C2752992A89549B6B8A8C0009345CD6F, ordinal: 0
2016-01-14 17:16:35,906 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C2752992A89549B6B8A8C0009345CD6F, mem on start (mb), free: 276, total: 720, max: 720
2016-01-14 17:16:35,906 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:35,906 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:35,907 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:35,907 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:35,907 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:35,908 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:35,908 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:35,908 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:35,908 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:35,909 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 755499008, TotalAvailable: 506184335, TotalRequested/TotalJVMHeap:0.14
2016-01-14 17:16:35,909 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, D2EA2B42DDDD4171B51B5F281EC37A43, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:35,909 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A79979D16F6A47C280295E4BB34D1669, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:35,909 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:35,910 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: D2EA2B42DDDD4171B51B5F281EC37A43
2016-01-14 17:16:35,910 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:35,910 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: D2EA2B42DDDD4171B51B5F281EC37A43
2016-01-14 17:16:35,910 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: D2EA2B42DDDD4171B51B5F281EC37A43 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:35,911 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:35,911 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:35,911 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:35,911 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791795705_0001_1_00_000000_0
2016-01-14 17:16:35,911 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:35,912 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@5da2ee56
2016-01-14 17:16:35,913 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@5da2ee56
2016-01-14 17:16:35,913 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:35,913 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:35,914 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:35,915 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: D2EA2B42DDDD4171B51B5F281EC37A43
2016-01-14 17:16:35,915 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 83EF745A19BD41E6A14ACC8BC2AA25A7
2016-01-14 17:16:35,915 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 83EF745A19BD41E6A14ACC8BC2AA25A7
2016-01-14 17:16:35,916 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@4aa38887; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@36587d59; comparator=cascading.tuple.hadoop.util.TupleComparator@6b8d79ee; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:35,916 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:35,916 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:35,916 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:35,945 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:35,945 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:35,945 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:35,946 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:35,946 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:35,948 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:35,948 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:35,949 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C2752992A89549B6B8A8C0009345CD6F, all 1 inputs ready in: 00:00:00.001
2016-01-14 17:16:35,951 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791795705_0001_1_00_000000_0
2016-01-14 17:16:35,951 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C2752992A89549B6B8A8C0009345CD6F, mem on close (mb), free: 380, total: 721, max: 721
2016-01-14 17:16:35,952 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:35,952 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:35,952 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:35,952 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:35,955 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:35 Completed running task attempt: attempt_1452791795705_0001_1_00_000000_0
2016-01-14 17:16:35,958 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791795705_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:35,959 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=15, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=756547584, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791795949, Process_Duration=2, Process_End_Time=1452791795951, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:35,959 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791795705_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791795705_0001_1_00, eventCount=1
2016-01-14 17:16:35,959 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:35,959 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:35,959 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:35,959 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791795705_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C2752992A89549B6B8A8C0009345CD6F, taskAttemptId=attempt_1452791795705_0001_1_00_000000_0, startTime=1452791795889, finishTime=1452791795959, timeTaken=70, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=15, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=756547584, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791795949, Process_Duration=2, Process_End_Time=1452791795951, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:35,960 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791795705_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:35,960 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791795705_0001_1_00_000000_0
2016-01-14 17:16:35,960 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:35,961 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791795705_0001_00_000001
2016-01-14 17:16:35,961 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:35,961 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791795705_0001_1][Event:TASK_FINISHED]: vertexName=C2752992A89549B6B8A8C0009345CD6F, taskId=task_1452791795705_0001_1_00_000000, startTime=1452791795889, finishTime=1452791795960, timeTaken=71, status=SUCCEEDED, successfulAttemptID=attempt_1452791795705_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=15, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=756547584, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791795949, Process_Duration=2, Process_End_Time=1452791795951, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:35,961 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791795705_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:35,961 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791795705_0001_1_00 [C2752992A89549B6B8A8C0009345CD6F] : 1
2016-01-14 17:16:35,962 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791795705_0001_1_00 [C2752992A89549B6B8A8C0009345CD6F]
2016-01-14 17:16:35,962 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:35,962 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791795705_0001_1][Event:VERTEX_FINISHED]: vertexName=C2752992A89549B6B8A8C0009345CD6F, vertexId=vertex_1452791795705_0001_1_00, initRequestedTime=1452791795868, initedTime=1452791795878, startRequestedTime=1452791795879, startedTime=1452791795879, finishTime=1452791795962, timeTaken=83, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=15, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=756547584, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791795949, Process_Duration=2, Process_End_Time=1452791795951, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791795889, firstTasksToStart=[ task_1452791795705_0001_1_00_000000 ], lastTaskFinishTime=1452791795959, lastTasksToFinish=[ task_1452791795705_0001_1_00_000000 ], minTaskDuration=70, maxTaskDuration=70, avgTaskDuration=70.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791795705_0001_1_00_000000 ], longestDurationTasks=[ task_1452791795705_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:35,962 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791795705_0001_1_00 [C2752992A89549B6B8A8C0009345CD6F] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:35,962 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791795705_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:35,963 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791795705_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791795705_0001_00_000001, stoppedTime=1452791795963, exitStatus=0
2016-01-14 17:16:35,963 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791795705_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:35,963 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791795705_0001_1_01 [A79979D16F6A47C280295E4BB34D1669] attempt: attempt_1452791795705_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:35,963 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: A79979D16F6A47C280295E4BB34D1669
2016-01-14 17:16:35,963 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791795705_0001_1_01 [A79979D16F6A47C280295E4BB34D1669]
2016-01-14 17:16:35,963 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791795705_0001_1_00 [C2752992A89549B6B8A8C0009345CD6F] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:35,963 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791795705_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:35,964 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791795705_0001_1][Event:TASK_STARTED]: vertexName=A79979D16F6A47C280295E4BB34D1669, taskId=task_1452791795705_0001_1_01_000000, scheduledTime=1452791795963, launchTime=1452791795963
2016-01-14 17:16:35,963 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:35,964 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791795705_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:35,964 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791795705_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:35,964 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:35,964 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791795705_0001_1_01 [A79979D16F6A47C280295E4BB34D1669]
2016-01-14 17:16:35,964 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [9B13F80AEFFB491CA058B9DE9CC0912D/FBFE8631A6944DB992E88A9EE5898757] every/(1/1) ...mblyplatform/0_UNKNOWN_-1, VertexName: A79979D16F6A47C280295E4BB34D1669, VertexParallelism: 1, TaskAttemptID:attempt_1452791795705_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C2752992A89549B6B8A8C0009345CD6F, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=FD8C91F533344213B34B61BE65032F43, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:35,965 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791795705_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:35,965 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:35,965 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791795705_0001_00_000002
2016-01-14 17:16:35,966 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791795705_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:35,967 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791795705_0001_00_000002
2016-01-14 17:16:35,967 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:35,967 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791795705_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:35,967 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:35,968 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791795705_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791795705_0001_00_000002, launchTime=1452791795967
2016-01-14 17:16:35,968 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791795705_0001_00_000002
2016-01-14 17:16:35,968 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791795705_0001_1_01_000000_0] to container: [container_1452791795705_0001_00_000002]
2016-01-14 17:16:35,968 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791795705_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:35,969 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791795705_0001_00_000002 given task: attempt_1452791795705_0001_1_01_000000_0
2016-01-14 17:16:35,969 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791795705_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791795705_0001_1_01_000000_0
2016-01-14 17:16:35,969 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791795705_0001_1_01_000000_0] started. Is using containerId: [container_1452791795705_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:35,969 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791795705_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A79979D16F6A47C280295E4BB34D1669, taskAttemptId=attempt_1452791795705_0001_1_01_000000_0, startTime=1452791795969, containerId=container_1452791795705_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791795705_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:35,969 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:35 Starting to run new task attempt: attempt_1452791795705_0001_1_01_000000_0
2016-01-14 17:16:35,969 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791795705_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:35,969 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791795705_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:35,969 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791795705_0001_1_01_000000_0
2016-01-14 17:16:35,969 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:35,970 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:35,970 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:35,970 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:35,970 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:35,970 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:35,970 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [9B13F80AEFFB491CA058B9DE9CC0912D/FBFE8631A6944DB992E88A9EE5898757] every/(1/1) ...mblyplatform/0_UNKNOWN_-1, VertexName: A79979D16F6A47C280295E4BB34D1669, VertexParallelism: 1, TaskAttemptID:attempt_1452791795705_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C2752992A89549B6B8A8C0009345CD6F, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=FD8C91F533344213B34B61BE65032F43, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:35,970 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=756547584
2016-01-14 17:16:35,971 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791795705_0001_1_01_000000_0
2016-01-14 17:16:35,971 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:35,971 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:35,971 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791795705_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:35,971 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791795705_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:35,972 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791795705_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:35,972 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791795705_0001_00_000001, Exiting
2016-01-14 17:16:35,972 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791795705_0001_00_000001
2016-01-14 17:16:35,972 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=FD8C91F533344213B34B61BE65032F43, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:35,972 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:35,972 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:35,972 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791795705_0001_00_000001
2016-01-14 17:16:35,972 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: FD8C91F533344213B34B61BE65032F43
2016-01-14 17:16:35,973 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:35,972 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C2752992A89549B6B8A8C0009345CD6F, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:35,973 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: FD8C91F533344213B34B61BE65032F43
2016-01-14 17:16:35,974 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:35,974 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C2752992A89549B6B8A8C0009345CD6F
2016-01-14 17:16:35,976 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 680892800, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:35,976 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:35,976 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 680892800, type: INPUT, componentVertexName: C2752992A89549B6B8A8C0009345CD6F
2016-01-14 17:16:35,977 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C2752992A89549B6B8A8C0009345CD6F
2016-01-14 17:16:35,978 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:35,978 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791795705_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:35,978 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791795705_0001_00_000001 completed successfully
2016-01-14 17:16:35,979 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791795705_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:35,982 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:35,982 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: FD8C91F533344213B34B61BE65032F43
2016-01-14 17:16:35,988 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A79979D16F6A47C280295E4BB34D1669, ordinal: 1
2016-01-14 17:16:35,988 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A79979D16F6A47C280295E4BB34D1669, mem on start (mb), free: 374, total: 721, max: 721
2016-01-14 17:16:35,988 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:35,988 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:35,988 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:35,988 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:35,989 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:35,990 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:35,990 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:35,990 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:35,991 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:35,991 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 680892800, TotalRequestedScaled: 6.285164307692308E8, TotalJVMHeap: 756547584, TotalAvailable: 506886881, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:35,991 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, FD8C91F533344213B34B61BE65032F43, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:35,992 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C2752992A89549B6B8A8C0009345CD6F, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=680892800, allocated=506886881
2016-01-14 17:16:35,992 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:35,993 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C2752992A89549B6B8A8C0009345CD6F
2016-01-14 17:16:35,995 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C2752992A89549B6B8A8C0009345CD6F being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:35,995 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:35,995 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:35,995 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:35,995 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:35,996 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: C2752992A89549B6B8A8C0009345CD6F with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:35,996 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=680892800, postMergeMem=0, RuntimeTotalAvailable=506886881. Updated to: ShuffleMem=506886881, postMergeMem: 0
2016-01-14 17:16:35,996 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=506886881, maxSingleShuffleLimit=126721720, mergeThreshold=456198176, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:35,996 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:35,997 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C2752992A89549B6B8A8C0009345CD6F
2016-01-14 17:16:35,997 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:35,997 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791795705_0001_1_01_000000_0
2016-01-14 17:16:35,998 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791795705_0001_1_00_000000_0_10147, runDuration: 56657, ]
2016-01-14 17:16:35,999 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791795705_0001_1_00_000000_0_10147] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791795999, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 17:16:35,999 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : C2752992A89549B6B8A8C0009345CD6F
2016-01-14 17:16:35,999 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:35,999 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [C2752992A89549B6B8A8C0009345CD6F] #25 in 1ms
2016-01-14 17:16:35,999 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [C2752992A89549B6B8A8C0009345CD6F] #25, status:false, isInterrupted:false
2016-01-14 17:16:36,000 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:36,000 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:36,000 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:36,000 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:36,000 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:36,000 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : C2752992A89549B6B8A8C0009345CD6F
2016-01-14 17:16:36,000 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:36,003 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:36,003 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 83EF745A19BD41E6A14ACC8BC2AA25A7
2016-01-14 17:16:36,004 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_-1"], id: FD8C91F533344213B34B61BE65032F43
2016-01-14 17:16:36,004 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_-1"]
2016-01-14 17:16:36,004 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 83EF745A19BD41E6A14ACC8BC2AA25A7, for 1 inputs
2016-01-14 17:16:36,004 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A79979D16F6A47C280295E4BB34D1669, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:36,005 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@1793bf94; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@650699a3
2016-01-14 17:16:36,011 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:36,011 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:36,011 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791795705_0001_1_01_000000_0
2016-01-14 17:16:36,012 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791795705_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:36,012 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791795705_0001_1_01_000000_0
2016-01-14 17:16:36,013 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A79979D16F6A47C280295E4BB34D1669, mem on close (mb), free: 371, total: 721, max: 721
2016-01-14 17:16:36,013 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: C2752992A89549B6B8A8C0009345CD6F
2016-01-14 17:16:36,013 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:36,014 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:36,014 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:36,014 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:36,014 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:36 Completed running task attempt: attempt_1452791795705_0001_1_01_000000_0
2016-01-14 17:16:36,016 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791795705_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:36,016 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=756547584, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=5, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791796004, Process_Duration=8, Process_End_Time=1452791796012, Read_Duration=4, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 17:16:36,017 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791795705_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A79979D16F6A47C280295E4BB34D1669, taskAttemptId=attempt_1452791795705_0001_1_01_000000_0, startTime=1452791795969, finishTime=1452791796016, timeTaken=47, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=756547584, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=5, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791796004, Process_Duration=8, Process_End_Time=1452791796012, Read_Duration=4, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:36,017 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:36,017 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:36,016 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:36,017 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791795705_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:36,017 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791795705_0001_1_01_000000_0
2016-01-14 17:16:36,017 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791795705_0001_00_000002
2016-01-14 17:16:36,017 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:36,018 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:36,018 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791795705_0001_1][Event:TASK_FINISHED]: vertexName=A79979D16F6A47C280295E4BB34D1669, taskId=task_1452791795705_0001_1_01_000000, startTime=1452791795969, finishTime=1452791796017, timeTaken=48, status=SUCCEEDED, successfulAttemptID=attempt_1452791795705_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=756547584, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=5, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791796004, Process_Duration=8, Process_End_Time=1452791796012, Read_Duration=4, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:36,018 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791795705_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:36,018 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791795705_0001_1_01 [A79979D16F6A47C280295E4BB34D1669] : 1
2016-01-14 17:16:36,018 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791795705_0001_1_01 [A79979D16F6A47C280295E4BB34D1669]
2016-01-14 17:16:36,019 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:36,019 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791795705_0001_1][Event:VERTEX_FINISHED]: vertexName=A79979D16F6A47C280295E4BB34D1669, vertexId=vertex_1452791795705_0001_1_01, initRequestedTime=1452791795869, initedTime=1452791795876, startRequestedTime=1452791795881, startedTime=1452791795881, finishTime=1452791796018, timeTaken=137, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=756547584, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=5, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791796004, Process_Duration=8, Process_End_Time=1452791796012, Read_Duration=4, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452791795969, firstTasksToStart=[ task_1452791795705_0001_1_01_000000 ], lastTaskFinishTime=1452791796016, lastTasksToFinish=[ task_1452791795705_0001_1_01_000000 ], minTaskDuration=47, maxTaskDuration=47, avgTaskDuration=47.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791795705_0001_1_01_000000 ], longestDurationTasks=[ task_1452791795705_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:36,019 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791795705_0001_1_01 [A79979D16F6A47C280295E4BB34D1669] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:36,019 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791795705_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:36,019 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791795705_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791795705_0001_00_000002, stoppedTime=1452791796019, exitStatus=0
2016-01-14 17:16:36,020 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791795705_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:36,020 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791795705_0001_1_01 [A79979D16F6A47C280295E4BB34D1669] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:36,020 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791795705_0001_1
2016-01-14 17:16:36,020 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791795705_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452791795705_0001_1
2016-01-14 17:16:36,020 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452791795705_0001_1_00 [C2752992A89549B6B8A8C0009345CD6F]
2016-01-14 17:16:36,020 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: FD8C91F533344213B34B61BE65032F43 for vertex: vertex_1452791795705_0001_1_01 [A79979D16F6A47C280295E4BB34D1669]
2016-01-14 17:16:36,020 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:36,028 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791795705_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 17:16:36,028 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791795705_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:36,028 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791795705_0001_00_000002, Exiting
2016-01-14 17:16:36,029 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791795705_0001_00_000002
2016-01-14 17:16:36,029 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791795705_0001_00_000002
2016-01-14 17:16:36,029 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791795705_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:36,029 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791795705_0001_00_000002 completed successfully
2016-01-14 17:16:36,030 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791795705_0001_1, queueSize=0
2016-01-14 17:16:36,030 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:36,031 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791795705_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791795705_0001_1, startTime=1452791795868, finishTime=1452791796027, timeTaken=159, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=465, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=15, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1513095168, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=5, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905583591953, Process_Duration=10, Process_End_Time=2905583591963, Read_Duration=5, Tuples_Read=40, Tuples_Written=40, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 17:16:36,031 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791795705_0001_1 finished with state: SUCCEEDED
2016-01-14 17:16:36,031 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791795705_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:36,031 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452791795705_0001_00_000002
2016-01-14 17:16:36,031 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791795705_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:16:36 Completed Dag: dag_1452791795705_0001_1
2016-01-14 17:16:36,032 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791795705_0001_1, dagState=SUCCEEDED
2016-01-14 17:16:36,032 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791795705_0001_1_post
2016-01-14 17:16:36,032 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:36,032 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:36,038 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...mblyplatform/0_UNKNOWN_-1, applicationId=application_1452791795705_0001
2016-01-14 17:16:36,039 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:36,039 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:36,040 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:36,040 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:36,040 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:36,040 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:36,040 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:36,041 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:36,041 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:36,041 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:36,042 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:36,042 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:36,042 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:36,047 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/9B13F80AEFFB491CA058B9DE9CC0912D/FBFE8631A6944DB992E88A9EE5898757/.tez/application_1452791795705_0001
2016-01-14 17:16:36,047 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.360
2016-01-14 17:16:36,113 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_UNKNOWN_-1
2016-01-14 17:16:36,133 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:36,133 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:36,158 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.002
2016-01-14 17:16:36,159 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.003
2016-01-14 17:16:36,159 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 17:16:36,160 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 17:16:36,160 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_UNKNOWN_'field'
2016-01-14 17:16:36,180 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:36,180 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:36,211 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.007
2016-01-14 17:16:36,218 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-01-14 17:16:36,218 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:36,219 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:36,219 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:36,232 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:36,240 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_RESULTS
2016-01-14 17:16:36,289 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:36,290 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:36,290 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_RESULTS"]
2016-01-14 17:16:36,290 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:36,290 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:36,290 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:36,292 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...latform/0_UNKNOWN_RESULTS
2016-01-14 17:16:36,292 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:36,309 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:36,313 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:36,314 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:36,314 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:36,323 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/C2F09C7CDE114AE5AF6CA0116C6A06EC/F81775A0464F411B8EBBF6917DA5E137/.tez/application_1452791796314_0001 doesn't exist and is created
2016-01-14 17:16:36,339 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:36,339 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:36,340 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/C2F09C7CDE114AE5AF6CA0116C6A06EC/F81775A0464F411B8EBBF6917DA5E137/.tez/application_1452791796314_0001
2016-01-14 17:16:36,342 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791796314_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:36,352 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:36,353 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:36,354 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:36,354 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:36,354 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/C2F09C7CDE114AE5AF6CA0116C6A06EC/F81775A0464F411B8EBBF6917DA5E137/.tez/application_1452791796314_0001/localmode-log-dir
2016-01-14 17:16:36,355 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/C2F09C7CDE114AE5AF6CA0116C6A06EC/F81775A0464F411B8EBBF6917DA5E137/.tez/application_1452791796314_0001/localmode-log-dir/history.txt.appattempt_1452791796314_0001_000000, maxErrors=10
2016-01-14 17:16:36,355 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:36,355 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791796314_0001_000000, appSubmitTime=1452791796342, launchTime=1452791796342
2016-01-14 17:16:36,358 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:36,361 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:36,361 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:57951
2016-01-14 17:16:36,362 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:36,362 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:36,363 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:36,365 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791796314_0001_000000, startTime=1452791796365
2016-01-14 17:16:36,365 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:36,365 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:36,387 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 17:16:36,387 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 17:16:36,440 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:36,440 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:36,440 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/0_UNKNOWN_RESULTS, applicationId=application_1452791796314_0001, dagName=[C2F09C7CDE114AE5AF6CA0116C6A06EC/F81775A0464F411B8EBBF6917DA5E137] every/(1/1) ...latform/0_UNKNOWN_RESULTS
2016-01-14 17:16:36,455 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [C2F09C7CDE114AE5AF6CA0116C6A06EC/F81775A0464F411B8EBBF6917DA5E137] every/(1/1) ...latform/0_UNKNOWN_RESULTS
2016-01-14 17:16:36,456 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791796314_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/C2F09C7CDE114AE5AF6CA0116C6A06EC/F81775A0464F411B8EBBF6917DA5E137/.tez/application_1452791796314_0001/localmode-log-dir/dag_1452791796314_0001_1.dot
2016-01-14 17:16:36,457 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/C2F09C7CDE114AE5AF6CA0116C6A06EC/F81775A0464F411B8EBBF6917DA5E137/.tez/application_1452791796314_0001/localmode-log-dir/dag_1452791796314_0001_1-tez-dag.pb.txt
2016-01-14 17:16:36,462 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791796314_0001_1
2016-01-14 17:16:36,462 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:36,462 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [C2F09C7CDE114AE5AF6CA0116C6A06EC/F81775A0464F411B8EBBF6917DA5E137] every/(1/1) ...latform/0_UNKNOWN_RESULTS
2016-01-14 17:16:36 Running Dag: dag_1452791796314_0001_1
2016-01-14 17:16:36,473 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791796314_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791796314_0001_1, submitTime=1452791796455
2016-01-14 17:16:36,473 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:36,474 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex B990BCCF97F5405397545E8E34C53315
2016-01-14 17:16:36,474 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:36,475 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:36,475 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791796314_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791796314_0001_1, initTime=1452791796474
2016-01-14 17:16:36,475 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791796314_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:36,476 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/0_UNKNOWN_RESULTS, applicationId=application_1452791796314_0001, dagName=[C2F09C7CDE114AE5AF6CA0116C6A06EC/F81775A0464F411B8EBBF6917DA5E137] every/(1/1) ...latform/0_UNKNOWN_RESULTS
2016-01-14 17:16:36,476 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791796314_0001_1][Event:DAG_STARTED]: dagID=dag_1452791796314_0001_1, startTime=1452791796476
2016-01-14 17:16:36,476 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:36,476 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:36,476 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791796314_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:36,476 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: DADF901A4051477AAE27BFBD30891181 : {D777224FA7674A9A95528BCF968F95CD={InputName=D777224FA7674A9A95528BCF968F95CD}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:36,476 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: D777224FA7674A9A95528BCF968F95CD, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:36,476 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791796314_0001_1_00 [DADF901A4051477AAE27BFBD30891181]
2016-01-14 17:16:36,477 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791796314_0001_1_00 [DADF901A4051477AAE27BFBD30891181]
2016-01-14 17:16:36,477 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791796314_0001_1_00 [DADF901A4051477AAE27BFBD30891181]
2016-01-14 17:16:36,477 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791796314_0001_1_00 [DADF901A4051477AAE27BFBD30891181]. Starting root input initializers: 1
2016-01-14 17:16:36,477 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791796314_0001_1_00 [DADF901A4051477AAE27BFBD30891181] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:36,477 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791796314_0001_1_01 [B990BCCF97F5405397545E8E34C53315]
2016-01-14 17:16:36,480 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:36,480 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791796314_0001_1_01 [B990BCCF97F5405397545E8E34C53315]
2016-01-14 17:16:36,480 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791796314_0001_1_01 [B990BCCF97F5405397545E8E34C53315]
2016-01-14 17:16:36,480 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791796314_0001_1_01 [B990BCCF97F5405397545E8E34C53315]
2016-01-14 17:16:36,480 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=942EF10FABA0410ABFE60F2C1D975755, vertexId=vertex_1452791796314_0001_1_01 [B990BCCF97F5405397545E8E34C53315], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:36,480 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=942EF10FABA0410ABFE60F2C1D975755, vertexId=vertex_1452791796314_0001_1_01 [B990BCCF97F5405397545E8E34C53315]
2016-01-14 17:16:36,481 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: D777224FA7674A9A95528BCF968F95CD on vertex vertex_1452791796314_0001_1_00 [DADF901A4051477AAE27BFBD30891181]
2016-01-14 17:16:36,482 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=942EF10FABA0410ABFE60F2C1D975755, vertexName=B990BCCF97F5405397545E8E34C53315, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:36,482 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=942EF10FABA0410ABFE60F2C1D975755, vertexId=vertex_1452791796314_0001_1_01 [B990BCCF97F5405397545E8E34C53315]
2016-01-14 17:16:36,483 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:36,483 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791796314_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B990BCCF97F5405397545E8E34C53315, vertexId=vertex_1452791796314_0001_1_01, initRequestedTime=1452791796477, initedTime=1452791796483, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:36,483 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791796314_0001_1_01 [B990BCCF97F5405397545E8E34C53315] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:36,501 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:36,502 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input D777224FA7674A9A95528BCF968F95CD asking for 1 tasks. Headroom: 717 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:36,502 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:36,503 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:36,504 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:36,505 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: D777224FA7674A9A95528BCF968F95CD on vertex vertex_1452791796314_0001_1_00 [DADF901A4051477AAE27BFBD30891181]
2016-01-14 17:16:36,505 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791796314_0001_1_00 [DADF901A4051477AAE27BFBD30891181] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:36,505 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {D777224FA7674A9A95528BCF968F95CD=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:36,505 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791796314_0001_1_00 [DADF901A4051477AAE27BFBD30891181] parallelism set to 1
2016-01-14 17:16:36,505 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791796314_0001_1_00 [DADF901A4051477AAE27BFBD30891181] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:36,506 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791796314_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791796314_0001_1_00, eventCount=1
2016-01-14 17:16:36,506 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:36,507 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791796314_0001_1][Event:VERTEX_INITIALIZED]: vertexName=DADF901A4051477AAE27BFBD30891181, vertexId=vertex_1452791796314_0001_1_00, initRequestedTime=1452791796476, initedTime=1452791796506, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:36,507 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791796314_0001_1_00 [DADF901A4051477AAE27BFBD30891181] with distanceFromRoot: 0
2016-01-14 17:16:36,507 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791796314_0001_1_00 [DADF901A4051477AAE27BFBD30891181] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:36,507 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in DADF901A4051477AAE27BFBD30891181
2016-01-14 17:16:36,507 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791796314_0001_1_00 [DADF901A4051477AAE27BFBD30891181]
2016-01-14 17:16:36,508 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:36,508 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791796314_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791796314_0001_1_00, startRequestedTime=1452791796507, startedTime=1452791796507
2016-01-14 17:16:36,508 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791796314_0001_1_00 [DADF901A4051477AAE27BFBD30891181] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:36,509 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:36,509 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791796314_0001_1][Event:TASK_STARTED]: vertexName=DADF901A4051477AAE27BFBD30891181, taskId=task_1452791796314_0001_1_00_000000, scheduledTime=1452791796509, launchTime=1452791796509
2016-01-14 17:16:36,509 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791796314_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:36,509 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791796314_0001_1_00 for vertex: vertex_1452791796314_0001_1_01 [B990BCCF97F5405397545E8E34C53315] numStartedSources: 1 numSources: 1
2016-01-14 17:16:36,510 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791796314_0001_1_01 [B990BCCF97F5405397545E8E34C53315] with distanceFromRoot: 1
2016-01-14 17:16:36,510 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791796314_0001_1_00 [DADF901A4051477AAE27BFBD30891181]
2016-01-14 17:16:36,510 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791796314_0001, with dag id: dag_1452791796314_0001_1
2016-01-14 17:16:36,510 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: DADF901A4051477AAE27BFBD30891181 in vertex: B990BCCF97F5405397545E8E34C53315
2016-01-14 17:16:36,511 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: B990BCCF97F5405397545E8E34C53315 with 1 source tasks and 1 pending tasks
2016-01-14 17:16:36,511 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:36,511 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791796314_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791796314_0001_1_01, startRequestedTime=1452791796510, startedTime=1452791796510
2016-01-14 17:16:36,512 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791796314_0001_1_01 [B990BCCF97F5405397545E8E34C53315] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:36,512 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [C2F09C7CDE114AE5AF6CA0116C6A06EC/F81775A0464F411B8EBBF6917DA5E137] every/(1/1) ...latform/0_UNKNOWN_RESULTS, VertexName: DADF901A4051477AAE27BFBD30891181, VertexParallelism: 1, TaskAttemptID:attempt_1452791796314_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D777224FA7674A9A95528BCF968F95CD, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B990BCCF97F5405397545E8E34C53315, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:36,512 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791796314_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:36,512 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:36,513 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:36,513 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:36,514 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:36,514 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791796314_0001_00_000001
2016-01-14 17:16:36,514 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791796314_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:36,515 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791796314_0001_00_000001
2016-01-14 17:16:36,515 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:36,516 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791796314_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:36,516 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:36,516 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791796314_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791796314_0001_00_000001, launchTime=1452791796516
2016-01-14 17:16:36,517 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791796314_0001_00_000001
2016-01-14 17:16:36,518 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791796314_0001_1_00_000000_0] to container: [container_1452791796314_0001_00_000001]
2016-01-14 17:16:36,518 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791796314_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:36,518 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791796314_0001_00_000001 given task: attempt_1452791796314_0001_1_00_000000_0
2016-01-14 17:16:36,518 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791796314_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791796314_0001_1_00_000000_0
2016-01-14 17:16:36 Starting to run new task attempt: attempt_1452791796314_0001_1_00_000000_0
2016-01-14 17:16:36,519 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791796314_0001_1_00_000000_0
2016-01-14 17:16:36,519 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791796314_0001_1_00_000000_0] started. Is using containerId: [container_1452791796314_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:36,519 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791796314_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=DADF901A4051477AAE27BFBD30891181, taskAttemptId=attempt_1452791796314_0001_1_00_000000_0, startTime=1452791796519, containerId=container_1452791796314_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791796314_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:36,519 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791796314_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:36,519 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791796314_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:36,519 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:36,519 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:36,520 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:36,520 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:36,520 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:36,520 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:36,520 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:36,520 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [C2F09C7CDE114AE5AF6CA0116C6A06EC/F81775A0464F411B8EBBF6917DA5E137] every/(1/1) ...latform/0_UNKNOWN_RESULTS, VertexName: DADF901A4051477AAE27BFBD30891181, VertexParallelism: 1, TaskAttemptID:attempt_1452791796314_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D777224FA7674A9A95528BCF968F95CD, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B990BCCF97F5405397545E8E34C53315, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:36,520 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=752353280
2016-01-14 17:16:36,521 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791796314_0001_1_00_000000_0
2016-01-14 17:16:36,521 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:36,521 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:36,522 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791796314_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:36,523 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=D777224FA7674A9A95528BCF968F95CD, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:36,523 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:36,523 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: D777224FA7674A9A95528BCF968F95CD
2016-01-14 17:16:36,523 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: D777224FA7674A9A95528BCF968F95CD
2016-01-14 17:16:36,524 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:36,524 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=B990BCCF97F5405397545E8E34C53315, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:36,524 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:36,525 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: B990BCCF97F5405397545E8E34C53315
2016-01-14 17:16:36,526 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:36,526 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: B990BCCF97F5405397545E8E34C53315
2016-01-14 17:16:36,526 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: B990BCCF97F5405397545E8E34C53315
2016-01-14 17:16:36,527 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:36,529 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:36,529 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: D777224FA7674A9A95528BCF968F95CD
2016-01-14 17:16:36,529 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: D777224FA7674A9A95528BCF968F95CD
2016-01-14 17:16:36,531 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: DADF901A4051477AAE27BFBD30891181, ordinal: 0
2016-01-14 17:16:36,531 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DADF901A4051477AAE27BFBD30891181, mem on start (mb), free: 469, total: 717, max: 717
2016-01-14 17:16:36,531 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:36,532 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:36,532 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:36,532 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:36,532 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:36,532 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:36,532 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:36,533 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:36,533 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:36,533 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 752353280, TotalAvailable: 504076697, TotalRequested/TotalJVMHeap:0.14
2016-01-14 17:16:36,534 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, D777224FA7674A9A95528BCF968F95CD, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:36,534 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, B990BCCF97F5405397545E8E34C53315, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:36,534 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:36,534 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: D777224FA7674A9A95528BCF968F95CD being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:36,534 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:36,535 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:36,535 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: D777224FA7674A9A95528BCF968F95CD
2016-01-14 17:16:36,535 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:36,535 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: D777224FA7674A9A95528BCF968F95CD
2016-01-14 17:16:36,536 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:36,536 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791796314_0001_1_00_000000_0
2016-01-14 17:16:36,538 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:36,538 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: D777224FA7674A9A95528BCF968F95CD
2016-01-14 17:16:36,538 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 53E6853B508B48699A2263EAABD11C3C
2016-01-14 17:16:36,538 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 53E6853B508B48699A2263EAABD11C3C
2016-01-14 17:16:36,539 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@6f64b318; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@3c2a25ca; comparator=cascading.tuple.hadoop.util.TupleComparator@73170fa1; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:36,539 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:36,539 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:36,539 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:36,540 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:36,542 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@6f034e0e
2016-01-14 17:16:36,544 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@6f034e0e
2016-01-14 17:16:36,544 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:36,544 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:36,554 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:36,554 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:36,554 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:36,554 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:36,554 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:36,555 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:36,555 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:36,556 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: DADF901A4051477AAE27BFBD30891181, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:36,558 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791796314_0001_1_00_000000_0
2016-01-14 17:16:36,559 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DADF901A4051477AAE27BFBD30891181, mem on close (mb), free: 368, total: 717, max: 717
2016-01-14 17:16:36,559 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:36,559 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:36,560 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:36,560 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:36,564 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:36 Completed running task attempt: attempt_1452791796314_0001_1_00_000000_0
2016-01-14 17:16:36,567 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791796314_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:36,567 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791796314_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791796314_0001_1_00, eventCount=1
2016-01-14 17:16:36,567 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791796314_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=DADF901A4051477AAE27BFBD30891181, taskAttemptId=attempt_1452791796314_0001_1_00_000000_0, startTime=1452791796519, finishTime=1452791796567, timeTaken=48, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=752353280, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791796556, Process_Duration=2, Process_End_Time=1452791796558, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:36,567 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791796314_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:36,568 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:36,568 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791796314_0001_1_00_000000_0
2016-01-14 17:16:36,568 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:36,568 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791796314_0001_1][Event:TASK_FINISHED]: vertexName=DADF901A4051477AAE27BFBD30891181, taskId=task_1452791796314_0001_1_00_000000, startTime=1452791796519, finishTime=1452791796568, timeTaken=49, status=SUCCEEDED, successfulAttemptID=attempt_1452791796314_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=752353280, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791796556, Process_Duration=2, Process_End_Time=1452791796558, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:36,568 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:36,568 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791796314_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:36,568 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791796314_0001_1_00 [DADF901A4051477AAE27BFBD30891181] : 1
2016-01-14 17:16:36,569 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791796314_0001_1_00 [DADF901A4051477AAE27BFBD30891181]
2016-01-14 17:16:36,569 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=752353280, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791796556, Process_Duration=2, Process_End_Time=1452791796558, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:36,569 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791796314_0001_1][Event:VERTEX_FINISHED]: vertexName=DADF901A4051477AAE27BFBD30891181, vertexId=vertex_1452791796314_0001_1_00, initRequestedTime=1452791796476, initedTime=1452791796506, startRequestedTime=1452791796507, startedTime=1452791796507, finishTime=1452791796569, timeTaken=62, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=752353280, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791796556, Process_Duration=2, Process_End_Time=1452791796558, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791796519, firstTasksToStart=[ task_1452791796314_0001_1_00_000000 ], lastTaskFinishTime=1452791796567, lastTasksToFinish=[ task_1452791796314_0001_1_00_000000 ], minTaskDuration=48, maxTaskDuration=48, avgTaskDuration=48.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791796314_0001_1_00_000000 ], longestDurationTasks=[ task_1452791796314_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:36,570 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:36,569 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:36,570 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:36,570 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791796314_0001_1_00 [DADF901A4051477AAE27BFBD30891181] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:36,570 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791796314_0001_00_000001
2016-01-14 17:16:36,570 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791796314_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:36,570 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791796314_0001_1_01 [B990BCCF97F5405397545E8E34C53315] attempt: attempt_1452791796314_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:36,570 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: B990BCCF97F5405397545E8E34C53315
2016-01-14 17:16:36,571 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791796314_0001_1_01 [B990BCCF97F5405397545E8E34C53315]
2016-01-14 17:16:36,571 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791796314_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791796314_0001_00_000001, stoppedTime=1452791796571, exitStatus=0
2016-01-14 17:16:36,571 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:36,571 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791796314_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:36,571 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791796314_0001_1_00 [DADF901A4051477AAE27BFBD30891181] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:36,571 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791796314_0001_1][Event:TASK_STARTED]: vertexName=B990BCCF97F5405397545E8E34C53315, taskId=task_1452791796314_0001_1_01_000000, scheduledTime=1452791796571, launchTime=1452791796571
2016-01-14 17:16:36,571 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:36,571 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791796314_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:36,572 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791796314_0001_1_01 [B990BCCF97F5405397545E8E34C53315]
2016-01-14 17:16:36,572 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791796314_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:36,572 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [C2F09C7CDE114AE5AF6CA0116C6A06EC/F81775A0464F411B8EBBF6917DA5E137] every/(1/1) ...latform/0_UNKNOWN_RESULTS, VertexName: B990BCCF97F5405397545E8E34C53315, VertexParallelism: 1, TaskAttemptID:attempt_1452791796314_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=DADF901A4051477AAE27BFBD30891181, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=942EF10FABA0410ABFE60F2C1D975755, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:36,572 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791796314_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:36,572 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791796314_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:36,573 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:36,573 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791796314_0001_00_000002
2016-01-14 17:16:36,573 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791796314_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:36,575 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791796314_0001_00_000002
2016-01-14 17:16:36,575 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:36,575 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791796314_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:36,576 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:36,576 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791796314_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791796314_0001_00_000002, launchTime=1452791796576
2016-01-14 17:16:36,577 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791796314_0001_00_000002
2016-01-14 17:16:36,577 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791796314_0001_1_01_000000_0] to container: [container_1452791796314_0001_00_000002]
2016-01-14 17:16:36,577 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791796314_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:36,577 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791796314_0001_00_000002 given task: attempt_1452791796314_0001_1_01_000000_0
2016-01-14 17:16:36,578 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791796314_0001_00_000002: 1 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791796314_0001_1_01_000000_0
2016-01-14 17:16:36,578 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791796314_0001_1_01_000000_0] started. Is using containerId: [container_1452791796314_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:36,578 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791796314_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B990BCCF97F5405397545E8E34C53315, taskAttemptId=attempt_1452791796314_0001_1_01_000000_0, startTime=1452791796577, containerId=container_1452791796314_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791796314_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:36,578 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:36,578 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791796314_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:36,578 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791796314_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:36 Starting to run new task attempt: attempt_1452791796314_0001_1_01_000000_0
2016-01-14 17:16:36,579 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791796314_0001_1_01_000000_0
2016-01-14 17:16:36,579 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:36,579 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:36,579 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:36,579 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:36,579 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:36,580 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:36,580 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [C2F09C7CDE114AE5AF6CA0116C6A06EC/F81775A0464F411B8EBBF6917DA5E137] every/(1/1) ...latform/0_UNKNOWN_RESULTS, VertexName: B990BCCF97F5405397545E8E34C53315, VertexParallelism: 1, TaskAttemptID:attempt_1452791796314_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=DADF901A4051477AAE27BFBD30891181, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=942EF10FABA0410ABFE60F2C1D975755, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:36,580 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=752353280
2016-01-14 17:16:36,581 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791796314_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:36,581 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791796314_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:36,581 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791796314_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:36,581 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791796314_0001_00_000001, Exiting
2016-01-14 17:16:36,581 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791796314_0001_00_000001
2016-01-14 17:16:36,581 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791796314_0001_00_000001
2016-01-14 17:16:36,581 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791796314_0001_1_01_000000_0
2016-01-14 17:16:36,581 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:36,582 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:36,582 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791796314_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:36,582 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:36,582 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=DADF901A4051477AAE27BFBD30891181, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:36,582 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791796314_0001_00_000001 completed successfully
2016-01-14 17:16:36,583 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791796314_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:36,583 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:36,583 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: DADF901A4051477AAE27BFBD30891181
2016-01-14 17:16:36,583 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=942EF10FABA0410ABFE60F2C1D975755, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:36,584 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:36,584 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 942EF10FABA0410ABFE60F2C1D975755
2016-01-14 17:16:36,584 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:36,584 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 942EF10FABA0410ABFE60F2C1D975755
2016-01-14 17:16:36,585 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 677117952, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:36,585 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:36,585 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:36,585 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 677117952, type: INPUT, componentVertexName: DADF901A4051477AAE27BFBD30891181
2016-01-14 17:16:36,586 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: DADF901A4051477AAE27BFBD30891181
2016-01-14 17:16:36,591 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B990BCCF97F5405397545E8E34C53315, ordinal: 1
2016-01-14 17:16:36,591 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B990BCCF97F5405397545E8E34C53315, mem on start (mb), free: 362, total: 717, max: 717
2016-01-14 17:16:36,591 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:36,591 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:36,591 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:36,595 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:36,595 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 942EF10FABA0410ABFE60F2C1D975755
2016-01-14 17:16:36,596 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:36,596 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:36,596 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:36,596 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:36,597 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:36,597 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:36,597 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 677117952, TotalRequestedScaled: 6.250319556923077E8, TotalJVMHeap: 752353280, TotalAvailable: 504076697, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:36,597 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 942EF10FABA0410ABFE60F2C1D975755, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:36,597 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, DADF901A4051477AAE27BFBD30891181, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=677117952, allocated=504076697
2016-01-14 17:16:36,597 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:36,598 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: DADF901A4051477AAE27BFBD30891181 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:36,598 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:36,598 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:36,598 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: DADF901A4051477AAE27BFBD30891181
2016-01-14 17:16:36,599 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:36,600 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:36,600 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: DADF901A4051477AAE27BFBD30891181 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:36,600 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=677117952, postMergeMem=0, RuntimeTotalAvailable=504076697. Updated to: ShuffleMem=504076697, postMergeMem: 0
2016-01-14 17:16:36,600 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=504076697, maxSingleShuffleLimit=126019176, mergeThreshold=453669024, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:36,600 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:36,601 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: DADF901A4051477AAE27BFBD30891181
2016-01-14 17:16:36,602 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:36,602 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791796314_0001_1_01_000000_0
2016-01-14 17:16:36,603 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791796314_0001_1_00_000000_0_10153, runDuration: 40518, ]
2016-01-14 17:16:36,604 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791796314_0001_1_00_000000_0_10153] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791796604, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 17:16:36,604 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : DADF901A4051477AAE27BFBD30891181
2016-01-14 17:16:36,604 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:36,604 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [DADF901A4051477AAE27BFBD30891181] #26 in 1ms
2016-01-14 17:16:36,605 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [DADF901A4051477AAE27BFBD30891181] #26, status:false, isInterrupted:false
2016-01-14 17:16:36,607 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:36,607 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:36,607 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:36,607 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:36,607 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:36,607 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : DADF901A4051477AAE27BFBD30891181
2016-01-14 17:16:36,607 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:36,608 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:36,608 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 53E6853B508B48699A2263EAABD11C3C
2016-01-14 17:16:36,609 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_RESULTS"], id: 942EF10FABA0410ABFE60F2C1D975755
2016-01-14 17:16:36,609 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_RESULTS"]
2016-01-14 17:16:36,609 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 53E6853B508B48699A2263EAABD11C3C, for 1 inputs
2016-01-14 17:16:36,609 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B990BCCF97F5405397545E8E34C53315, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:36,610 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@56564a1f; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@3b1c6581
2016-01-14 17:16:36,614 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:36,614 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:36,614 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791796314_0001_1_01_000000_0
2016-01-14 17:16:36,614 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791796314_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:36,615 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791796314_0001_1_01_000000_0
2016-01-14 17:16:36,615 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B990BCCF97F5405397545E8E34C53315, mem on close (mb), free: 359, total: 717, max: 717
2016-01-14 17:16:36,615 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: DADF901A4051477AAE27BFBD30891181
2016-01-14 17:16:36,615 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:36,615 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:36,615 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:36,615 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:36,615 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:36 Completed running task attempt: attempt_1452791796314_0001_1_01_000000_0
2016-01-14 17:16:36,616 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791796314_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:36,616 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=752353280, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791796609, Process_Duration=5, Process_End_Time=1452791796614, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 17:16:36,617 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:36,617 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:36,617 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:36,617 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791796314_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B990BCCF97F5405397545E8E34C53315, taskAttemptId=attempt_1452791796314_0001_1_01_000000_0, startTime=1452791796577, finishTime=1452791796616, timeTaken=39, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=752353280, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791796609, Process_Duration=5, Process_End_Time=1452791796614, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:36,617 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791796314_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:36,617 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:36,617 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791796314_0001_00_000002
2016-01-14 17:16:36,617 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791796314_0001_1_01_000000_0
2016-01-14 17:16:36,618 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:36,618 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791796314_0001_1][Event:TASK_FINISHED]: vertexName=B990BCCF97F5405397545E8E34C53315, taskId=task_1452791796314_0001_1_01_000000, startTime=1452791796577, finishTime=1452791796618, timeTaken=41, status=SUCCEEDED, successfulAttemptID=attempt_1452791796314_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=752353280, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791796609, Process_Duration=5, Process_End_Time=1452791796614, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:36,618 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791796314_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:36,618 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791796314_0001_1_01 [B990BCCF97F5405397545E8E34C53315] : 1
2016-01-14 17:16:36,618 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791796314_0001_1_01 [B990BCCF97F5405397545E8E34C53315]
2016-01-14 17:16:36,619 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:36,619 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791796314_0001_1][Event:VERTEX_FINISHED]: vertexName=B990BCCF97F5405397545E8E34C53315, vertexId=vertex_1452791796314_0001_1_01, initRequestedTime=1452791796477, initedTime=1452791796483, startRequestedTime=1452791796510, startedTime=1452791796510, finishTime=1452791796618, timeTaken=108, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=752353280, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791796609, Process_Duration=5, Process_End_Time=1452791796614, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452791796577, firstTasksToStart=[ task_1452791796314_0001_1_01_000000 ], lastTaskFinishTime=1452791796616, lastTasksToFinish=[ task_1452791796314_0001_1_01_000000 ], minTaskDuration=39, maxTaskDuration=39, avgTaskDuration=39.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791796314_0001_1_01_000000 ], longestDurationTasks=[ task_1452791796314_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:36,619 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791796314_0001_1_01 [B990BCCF97F5405397545E8E34C53315] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:36,619 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791796314_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:36,620 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791796314_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791796314_0001_00_000002, stoppedTime=1452791796620, exitStatus=0
2016-01-14 17:16:36,620 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791796314_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:36,620 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791796314_0001_1_01 [B990BCCF97F5405397545E8E34C53315] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:36,620 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791796314_0001_1
2016-01-14 17:16:36,620 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791796314_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452791796314_0001_1
2016-01-14 17:16:36,620 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 942EF10FABA0410ABFE60F2C1D975755 for vertex: vertex_1452791796314_0001_1_01 [B990BCCF97F5405397545E8E34C53315]
2016-01-14 17:16:36,621 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:36,627 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452791796314_0001_1_00 [DADF901A4051477AAE27BFBD30891181]
2016-01-14 17:16:36,629 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791796314_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 17:16:36,629 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791796314_0001_1, queueSize=0
2016-01-14 17:16:36,629 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791796314_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:36,629 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791796314_0001_00_000002, Exiting
2016-01-14 17:16:36,629 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791796314_0001_00_000002
2016-01-14 17:16:36,630 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791796314_0001_00_000002
2016-01-14 17:16:36,630 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791796314_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:36,630 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791796314_0001_00_000002 completed successfully
2016-01-14 17:16:36,630 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:36,631 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791796314_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791796314_0001_1, startTime=1452791796476, finishTime=1452791796627, timeTaken=151, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=465, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1504706560, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905583593165, Process_Duration=7, Process_End_Time=2905583593172, Read_Duration=3, Tuples_Read=40, Tuples_Written=40, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 17:16:36,631 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791796314_0001_1 finished with state: SUCCEEDED
2016-01-14 17:16:36,631 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791796314_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:36,631 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791796314_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:16:36,631 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452791796314_0001_00_000002
2016-01-14 17:16:36 Completed Dag: dag_1452791796314_0001_1
2016-01-14 17:16:36,633 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791796314_0001_1, dagState=SUCCEEDED
2016-01-14 17:16:36,633 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791796314_0001_1_post
2016-01-14 17:16:36,633 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:36,633 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:36,637 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/0_UNKNOWN_RESULTS, applicationId=application_1452791796314_0001
2016-01-14 17:16:36,639 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:36,639 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:36,639 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:36,639 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:36,639 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:36,639 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:36,640 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:36,640 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:36,640 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:36,640 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:36,640 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:36,641 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:36,641 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:36,646 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/C2F09C7CDE114AE5AF6CA0116C6A06EC/F81775A0464F411B8EBBF6917DA5E137/.tez/application_1452791796314_0001
2016-01-14 17:16:36,647 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.358
2016-01-14 17:16:36,711 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_UNKNOWN_RESULTS
2016-01-14 17:16:36,733 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:36,733 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:36,764 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.007
2016-01-14 17:16:36,775 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.018
2016-01-14 17:16:36,775 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:36,775 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:36,776 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:36,789 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:36,795 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_ALL
2016-01-14 17:16:36,836 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:36,836 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:36,837 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_ALL"]
2016-01-14 17:16:36,837 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:36,837 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:36,837 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:36,838 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...blyplatform/0_UNKNOWN_ALL
2016-01-14 17:16:36,838 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:36,855 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:36,860 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:36,860 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:36,861 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:36,870 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/53D6E9FE022F4473909CEBB0DC81DF55/C676A7E2584F4A6D86059FB08537B53C/.tez/application_1452791796860_0001 doesn't exist and is created
2016-01-14 17:16:36,889 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:36,890 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:36,890 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/53D6E9FE022F4473909CEBB0DC81DF55/C676A7E2584F4A6D86059FB08537B53C/.tez/application_1452791796860_0001
2016-01-14 17:16:36,893 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791796860_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:36,903 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:36,904 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:36,905 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:36,905 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:36,906 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/53D6E9FE022F4473909CEBB0DC81DF55/C676A7E2584F4A6D86059FB08537B53C/.tez/application_1452791796860_0001/localmode-log-dir
2016-01-14 17:16:36,906 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/53D6E9FE022F4473909CEBB0DC81DF55/C676A7E2584F4A6D86059FB08537B53C/.tez/application_1452791796860_0001/localmode-log-dir/history.txt.appattempt_1452791796860_0001_000000, maxErrors=10
2016-01-14 17:16:36,906 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:36,907 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791796860_0001_000000, appSubmitTime=1452791796893, launchTime=1452791796893
2016-01-14 17:16:36,911 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:36,912 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:55733
2016-01-14 17:16:36,913 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:36,915 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:36,915 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:36,920 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:36,922 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791796860_0001_000000, startTime=1452791796922
2016-01-14 17:16:36,922 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:36,922 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:36,990 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:36,990 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:36,990 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...blyplatform/0_UNKNOWN_ALL, applicationId=application_1452791796860_0001, dagName=[53D6E9FE022F4473909CEBB0DC81DF55/C676A7E2584F4A6D86059FB08537B53C] every/(1/1) ...blyplatform/0_UNKNOWN_ALL
2016-01-14 17:16:37,006 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [53D6E9FE022F4473909CEBB0DC81DF55/C676A7E2584F4A6D86059FB08537B53C] every/(1/1) ...blyplatform/0_UNKNOWN_ALL
2016-01-14 17:16:37,007 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791796860_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/53D6E9FE022F4473909CEBB0DC81DF55/C676A7E2584F4A6D86059FB08537B53C/.tez/application_1452791796860_0001/localmode-log-dir/dag_1452791796860_0001_1.dot
2016-01-14 17:16:37,007 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/53D6E9FE022F4473909CEBB0DC81DF55/C676A7E2584F4A6D86059FB08537B53C/.tez/application_1452791796860_0001/localmode-log-dir/dag_1452791796860_0001_1-tez-dag.pb.txt
2016-01-14 17:16:37,009 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 17:16:37,009 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 17:16:37,014 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791796860_0001_1
2016-01-14 17:16:37,015 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:37,015 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [53D6E9FE022F4473909CEBB0DC81DF55/C676A7E2584F4A6D86059FB08537B53C] every/(1/1) ...blyplatform/0_UNKNOWN_ALL
2016-01-14 17:16:37 Running Dag: dag_1452791796860_0001_1
2016-01-14 17:16:37,026 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791796860_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791796860_0001_1, submitTime=1452791797006
2016-01-14 17:16:37,026 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:37,027 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex A620C68CD0354BF5B589522118A6B831
2016-01-14 17:16:37,027 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:37,028 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791796860_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791796860_0001_1, initTime=1452791797026
2016-01-14 17:16:37,028 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:37,028 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791796860_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:37,029 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791796860_0001_1][Event:DAG_STARTED]: dagID=dag_1452791796860_0001_1, startTime=1452791797028
2016-01-14 17:16:37,029 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:37,029 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791796860_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:37,029 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:37,029 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: ABA83F29D3C94F2F83BD33F06F37CF83 : {C1301CA1D61045A0BEFFB7798ABF39ED={InputName=C1301CA1D61045A0BEFFB7798ABF39ED}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:37,029 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...blyplatform/0_UNKNOWN_ALL, applicationId=application_1452791796860_0001, dagName=[53D6E9FE022F4473909CEBB0DC81DF55/C676A7E2584F4A6D86059FB08537B53C] every/(1/1) ...blyplatform/0_UNKNOWN_ALL
2016-01-14 17:16:37,029 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: C1301CA1D61045A0BEFFB7798ABF39ED, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:37,029 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791796860_0001_1_00 [ABA83F29D3C94F2F83BD33F06F37CF83]
2016-01-14 17:16:37,029 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791796860_0001_1_00 [ABA83F29D3C94F2F83BD33F06F37CF83]
2016-01-14 17:16:37,029 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791796860_0001_1_00 [ABA83F29D3C94F2F83BD33F06F37CF83]
2016-01-14 17:16:37,030 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791796860_0001_1_00 [ABA83F29D3C94F2F83BD33F06F37CF83]. Starting root input initializers: 1
2016-01-14 17:16:37,030 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791796860_0001_1_00 [ABA83F29D3C94F2F83BD33F06F37CF83] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:37,030 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791796860_0001_1_01 [A620C68CD0354BF5B589522118A6B831]
2016-01-14 17:16:37,030 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: C1301CA1D61045A0BEFFB7798ABF39ED on vertex vertex_1452791796860_0001_1_00 [ABA83F29D3C94F2F83BD33F06F37CF83]
2016-01-14 17:16:37,032 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:37,032 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input C1301CA1D61045A0BEFFB7798ABF39ED asking for 1 tasks. Headroom: 717 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:37,033 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:37,033 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:37,034 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:37,034 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791796860_0001_1_01 [A620C68CD0354BF5B589522118A6B831]
2016-01-14 17:16:37,034 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791796860_0001_1_01 [A620C68CD0354BF5B589522118A6B831]
2016-01-14 17:16:37,034 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791796860_0001_1_01 [A620C68CD0354BF5B589522118A6B831]
2016-01-14 17:16:37,034 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=0B25CD57BF164404AAE15C99804F5A98, vertexId=vertex_1452791796860_0001_1_01 [A620C68CD0354BF5B589522118A6B831], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:37,034 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=0B25CD57BF164404AAE15C99804F5A98, vertexId=vertex_1452791796860_0001_1_01 [A620C68CD0354BF5B589522118A6B831]
2016-01-14 17:16:37,034 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:37,035 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: C1301CA1D61045A0BEFFB7798ABF39ED on vertex vertex_1452791796860_0001_1_00 [ABA83F29D3C94F2F83BD33F06F37CF83]
2016-01-14 17:16:37,036 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=0B25CD57BF164404AAE15C99804F5A98, vertexName=A620C68CD0354BF5B589522118A6B831, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:37,036 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=0B25CD57BF164404AAE15C99804F5A98, vertexId=vertex_1452791796860_0001_1_01 [A620C68CD0354BF5B589522118A6B831]
2016-01-14 17:16:37,037 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:37,037 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791796860_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A620C68CD0354BF5B589522118A6B831, vertexId=vertex_1452791796860_0001_1_01, initRequestedTime=1452791797030, initedTime=1452791797037, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:37,037 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791796860_0001_1_01 [A620C68CD0354BF5B589522118A6B831] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:37,038 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791796860_0001_1_00 [ABA83F29D3C94F2F83BD33F06F37CF83] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:37,038 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {C1301CA1D61045A0BEFFB7798ABF39ED=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:37,038 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791796860_0001_1_00 [ABA83F29D3C94F2F83BD33F06F37CF83] parallelism set to 1
2016-01-14 17:16:37,038 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791796860_0001_1_00 [ABA83F29D3C94F2F83BD33F06F37CF83] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:37,038 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791796860_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791796860_0001_1_00, eventCount=1
2016-01-14 17:16:37,038 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:37,038 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791796860_0001_1][Event:VERTEX_INITIALIZED]: vertexName=ABA83F29D3C94F2F83BD33F06F37CF83, vertexId=vertex_1452791796860_0001_1_00, initRequestedTime=1452791797029, initedTime=1452791797038, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:37,039 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791796860_0001_1_00 [ABA83F29D3C94F2F83BD33F06F37CF83] with distanceFromRoot: 0
2016-01-14 17:16:37,039 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791796860_0001_1_00 [ABA83F29D3C94F2F83BD33F06F37CF83] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:37,039 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in ABA83F29D3C94F2F83BD33F06F37CF83
2016-01-14 17:16:37,039 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791796860_0001_1_00 [ABA83F29D3C94F2F83BD33F06F37CF83]
2016-01-14 17:16:37,039 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:37,040 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791796860_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791796860_0001_1_00, startRequestedTime=1452791797039, startedTime=1452791797039
2016-01-14 17:16:37,040 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791796860_0001_1_00 [ABA83F29D3C94F2F83BD33F06F37CF83] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:37,040 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:37,040 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791796860_0001_1][Event:TASK_STARTED]: vertexName=ABA83F29D3C94F2F83BD33F06F37CF83, taskId=task_1452791796860_0001_1_00_000000, scheduledTime=1452791797040, launchTime=1452791797040
2016-01-14 17:16:37,041 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791796860_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:37,041 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791796860_0001_1_00 for vertex: vertex_1452791796860_0001_1_01 [A620C68CD0354BF5B589522118A6B831] numStartedSources: 1 numSources: 1
2016-01-14 17:16:37,041 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791796860_0001_1_01 [A620C68CD0354BF5B589522118A6B831] with distanceFromRoot: 1
2016-01-14 17:16:37,041 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791796860_0001_1_00 [ABA83F29D3C94F2F83BD33F06F37CF83]
2016-01-14 17:16:37,041 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: ABA83F29D3C94F2F83BD33F06F37CF83 in vertex: A620C68CD0354BF5B589522118A6B831
2016-01-14 17:16:37,042 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: A620C68CD0354BF5B589522118A6B831 with 1 source tasks and 1 pending tasks
2016-01-14 17:16:37,042 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:37,042 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791796860_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791796860_0001_1_01, startRequestedTime=1452791797041, startedTime=1452791797041
2016-01-14 17:16:37,042 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791796860_0001_1_01 [A620C68CD0354BF5B589522118A6B831] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:37,043 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [53D6E9FE022F4473909CEBB0DC81DF55/C676A7E2584F4A6D86059FB08537B53C] every/(1/1) ...blyplatform/0_UNKNOWN_ALL, VertexName: ABA83F29D3C94F2F83BD33F06F37CF83, VertexParallelism: 1, TaskAttemptID:attempt_1452791796860_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C1301CA1D61045A0BEFFB7798ABF39ED, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A620C68CD0354BF5B589522118A6B831, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:37,043 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791796860_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:37,043 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:37,044 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:37,044 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:37,044 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:37,044 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791796860_0001_00_000001
2016-01-14 17:16:37,045 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791796860_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:37,045 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791796860_0001, with dag id: dag_1452791796860_0001_1
2016-01-14 17:16:37,046 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791796860_0001_00_000001
2016-01-14 17:16:37,046 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:37,047 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791796860_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:37,047 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:37,047 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791796860_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791796860_0001_00_000001, launchTime=1452791797047
2016-01-14 17:16:37,048 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791796860_0001_00_000001
2016-01-14 17:16:37,048 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791796860_0001_1_00_000000_0] to container: [container_1452791796860_0001_00_000001]
2016-01-14 17:16:37,048 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791796860_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:37,049 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791796860_0001_1_00_000000_0] started. Is using containerId: [container_1452791796860_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:37,049 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791796860_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=ABA83F29D3C94F2F83BD33F06F37CF83, taskAttemptId=attempt_1452791796860_0001_1_00_000000_0, startTime=1452791797048, containerId=container_1452791796860_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791796860_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:37,049 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:37,049 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791796860_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:37,049 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791796860_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:37,049 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791796860_0001_00_000001 given task: attempt_1452791796860_0001_1_00_000000_0
2016-01-14 17:16:37,049 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791796860_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791796860_0001_1_00_000000_0
2016-01-14 17:16:37 Starting to run new task attempt: attempt_1452791796860_0001_1_00_000000_0
2016-01-14 17:16:37,050 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791796860_0001_1_00_000000_0
2016-01-14 17:16:37,050 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:37,050 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:37,050 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:37,050 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:37,050 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:37,050 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:37,050 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [53D6E9FE022F4473909CEBB0DC81DF55/C676A7E2584F4A6D86059FB08537B53C] every/(1/1) ...blyplatform/0_UNKNOWN_ALL, VertexName: ABA83F29D3C94F2F83BD33F06F37CF83, VertexParallelism: 1, TaskAttemptID:attempt_1452791796860_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C1301CA1D61045A0BEFFB7798ABF39ED, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A620C68CD0354BF5B589522118A6B831, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:37,050 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=752353280
2016-01-14 17:16:37,052 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791796860_0001_1_00_000000_0
2016-01-14 17:16:37,052 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:37,052 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:37,053 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791796860_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:37,053 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:37,053 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C1301CA1D61045A0BEFFB7798ABF39ED, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:37,054 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:37,054 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C1301CA1D61045A0BEFFB7798ABF39ED
2016-01-14 17:16:37,054 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: C1301CA1D61045A0BEFFB7798ABF39ED
2016-01-14 17:16:37,056 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:37,056 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:37,056 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: C1301CA1D61045A0BEFFB7798ABF39ED
2016-01-14 17:16:37,056 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C1301CA1D61045A0BEFFB7798ABF39ED
2016-01-14 17:16:37,056 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A620C68CD0354BF5B589522118A6B831, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:37,057 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:37,057 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A620C68CD0354BF5B589522118A6B831
2016-01-14 17:16:37,058 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:37,058 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: A620C68CD0354BF5B589522118A6B831
2016-01-14 17:16:37,059 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A620C68CD0354BF5B589522118A6B831
2016-01-14 17:16:37,062 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: ABA83F29D3C94F2F83BD33F06F37CF83, ordinal: 0
2016-01-14 17:16:37,062 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: ABA83F29D3C94F2F83BD33F06F37CF83, mem on start (mb), free: 268, total: 717, max: 717
2016-01-14 17:16:37,062 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:37,062 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:37,063 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:37,063 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:37,063 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:37,063 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:37,064 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:37,064 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:37,064 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:37,064 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 752353280, TotalAvailable: 504076697, TotalRequested/TotalJVMHeap:0.14
2016-01-14 17:16:37,065 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C1301CA1D61045A0BEFFB7798ABF39ED, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:37,065 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A620C68CD0354BF5B589522118A6B831, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:37,065 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:37,065 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C1301CA1D61045A0BEFFB7798ABF39ED being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:37,065 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C1301CA1D61045A0BEFFB7798ABF39ED
2016-01-14 17:16:37,066 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:37,066 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C1301CA1D61045A0BEFFB7798ABF39ED
2016-01-14 17:16:37,066 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:37,066 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:37,066 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:37,067 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791796860_0001_1_00_000000_0
2016-01-14 17:16:37,067 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:37,067 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@4ec35eb5
2016-01-14 17:16:37,085 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@4ec35eb5
2016-01-14 17:16:37,086 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:37,086 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:37,087 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:37,088 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: C1301CA1D61045A0BEFFB7798ABF39ED
2016-01-14 17:16:37,088 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 7F44C4AD18454752916DC8FC0F06C50B
2016-01-14 17:16:37,088 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 7F44C4AD18454752916DC8FC0F06C50B
2016-01-14 17:16:37,089 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@6ab494c6; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@759389e4; comparator=cascading.tuple.hadoop.util.TupleComparator@2942140a; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:37,089 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:37,089 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:37,089 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:37,102 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:37,102 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:37,102 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:37,102 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:37,102 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:37,102 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:37,102 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:37,103 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: ABA83F29D3C94F2F83BD33F06F37CF83, all 1 inputs ready in: 00:00:00.001
2016-01-14 17:16:37,105 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791796860_0001_1_00_000000_0
2016-01-14 17:16:37,105 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: ABA83F29D3C94F2F83BD33F06F37CF83, mem on close (mb), free: 375, total: 718, max: 718
2016-01-14 17:16:37,105 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:37,105 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:37,105 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:37,106 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:37,108 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:37 Completed running task attempt: attempt_1452791796860_0001_1_00_000000_0
2016-01-14 17:16:37,110 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791796860_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:37,111 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=17, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=753401856, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791797103, Process_Duration=2, Process_End_Time=1452791797105, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:37,111 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791796860_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791796860_0001_1_00, eventCount=1
2016-01-14 17:16:37,111 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:37,111 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791796860_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=ABA83F29D3C94F2F83BD33F06F37CF83, taskAttemptId=attempt_1452791796860_0001_1_00_000000_0, startTime=1452791797048, finishTime=1452791797111, timeTaken=63, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=17, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=753401856, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791797103, Process_Duration=2, Process_End_Time=1452791797105, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:37,111 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791796860_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:37,112 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791796860_0001_1_00_000000_0
2016-01-14 17:16:37,112 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:37,112 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791796860_0001_1][Event:TASK_FINISHED]: vertexName=ABA83F29D3C94F2F83BD33F06F37CF83, taskId=task_1452791796860_0001_1_00_000000, startTime=1452791797048, finishTime=1452791797112, timeTaken=64, status=SUCCEEDED, successfulAttemptID=attempt_1452791796860_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=17, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=753401856, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791797103, Process_Duration=2, Process_End_Time=1452791797105, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:37,112 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791796860_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:37,112 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791796860_0001_1_00 [ABA83F29D3C94F2F83BD33F06F37CF83] : 1
2016-01-14 17:16:37,112 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791796860_0001_1_00 [ABA83F29D3C94F2F83BD33F06F37CF83]
2016-01-14 17:16:37,113 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:37,113 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791796860_0001_1][Event:VERTEX_FINISHED]: vertexName=ABA83F29D3C94F2F83BD33F06F37CF83, vertexId=vertex_1452791796860_0001_1_00, initRequestedTime=1452791797029, initedTime=1452791797038, startRequestedTime=1452791797039, startedTime=1452791797039, finishTime=1452791797112, timeTaken=73, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=17, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=753401856, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791797103, Process_Duration=2, Process_End_Time=1452791797105, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791797048, firstTasksToStart=[ task_1452791796860_0001_1_00_000000 ], lastTaskFinishTime=1452791797111, lastTasksToFinish=[ task_1452791796860_0001_1_00_000000 ], minTaskDuration=63, maxTaskDuration=63, avgTaskDuration=63.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791796860_0001_1_00_000000 ], longestDurationTasks=[ task_1452791796860_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:37,113 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791796860_0001_1_00 [ABA83F29D3C94F2F83BD33F06F37CF83] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:37,113 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791796860_0001_1_01 [A620C68CD0354BF5B589522118A6B831] attempt: attempt_1452791796860_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:37,113 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: A620C68CD0354BF5B589522118A6B831
2016-01-14 17:16:37,113 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791796860_0001_1_01 [A620C68CD0354BF5B589522118A6B831]
2016-01-14 17:16:37,113 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791796860_0001_1_00 [ABA83F29D3C94F2F83BD33F06F37CF83] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:37,113 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:37,114 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791796860_0001_1][Event:TASK_STARTED]: vertexName=A620C68CD0354BF5B589522118A6B831, taskId=task_1452791796860_0001_1_01_000000, scheduledTime=1452791797114, launchTime=1452791797114
2016-01-14 17:16:37,114 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:37,114 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791796860_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:37,114 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791796860_0001_1_01 [A620C68CD0354BF5B589522118A6B831]
2016-01-14 17:16:37,114 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:37,115 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:37,115 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791796860_0001_00_000001
2016-01-14 17:16:37,115 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791796860_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:37,115 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791796860_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791796860_0001_00_000001, stoppedTime=1452791797115, exitStatus=0
2016-01-14 17:16:37,115 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791796860_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:37,115 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [53D6E9FE022F4473909CEBB0DC81DF55/C676A7E2584F4A6D86059FB08537B53C] every/(1/1) ...blyplatform/0_UNKNOWN_ALL, VertexName: A620C68CD0354BF5B589522118A6B831, VertexParallelism: 1, TaskAttemptID:attempt_1452791796860_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=ABA83F29D3C94F2F83BD33F06F37CF83, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=0B25CD57BF164404AAE15C99804F5A98, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:37,115 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791796860_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:37,115 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:37,116 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791796860_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:37,116 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:37,116 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791796860_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:37,116 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791796860_0001_00_000002
2016-01-14 17:16:37,116 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791796860_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:37,115 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(221)) - No more tasks for container with id : container_1452791796860_0001_00_000001. Asking it to die
2016-01-14 17:16:37,117 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791796860_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:37,117 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791796860_0001_00_000001, Exiting
2016-01-14 17:16:37,117 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791796860_0001_00_000001
2016-01-14 17:16:37,117 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791796860_0001_00_000001
2016-01-14 17:16:37,118 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791796860_0001_00_000002
2016-01-14 17:16:37,119 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 17:16:37,118 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791796860_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:37,121 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791796860_0001_00_000001 completed successfully
2016-01-14 17:16:37,121 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791796860_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:37,122 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791796860_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:37,122 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:37,123 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791796860_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791796860_0001_00_000002, launchTime=1452791797122
2016-01-14 17:16:37,123 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791796860_0001_00_000002
2016-01-14 17:16:37,124 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791796860_0001_1_01_000000_0] to container: [container_1452791796860_0001_00_000002]
2016-01-14 17:16:37,124 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791796860_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:37,125 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791796860_0001_1_01_000000_0] started. Is using containerId: [container_1452791796860_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:37,125 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791796860_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A620C68CD0354BF5B589522118A6B831, taskAttemptId=attempt_1452791796860_0001_1_01_000000_0, startTime=1452791797124, containerId=container_1452791796860_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791796860_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:37,125 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791796860_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:37,125 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791796860_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:37,125 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:37,126 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791796860_0001_00_000002 given task: attempt_1452791796860_0001_1_01_000000_0
2016-01-14 17:16:37,126 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791796860_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791796860_0001_1_01_000000_0
2016-01-14 17:16:37 Starting to run new task attempt: attempt_1452791796860_0001_1_01_000000_0
2016-01-14 17:16:37,126 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791796860_0001_1_01_000000_0
2016-01-14 17:16:37,126 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:37,126 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:37,126 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:37,127 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:37,127 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:37,127 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:37,127 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [53D6E9FE022F4473909CEBB0DC81DF55/C676A7E2584F4A6D86059FB08537B53C] every/(1/1) ...blyplatform/0_UNKNOWN_ALL, VertexName: A620C68CD0354BF5B589522118A6B831, VertexParallelism: 1, TaskAttemptID:attempt_1452791796860_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=ABA83F29D3C94F2F83BD33F06F37CF83, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=0B25CD57BF164404AAE15C99804F5A98, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:37,127 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=753401856
2016-01-14 17:16:37,128 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791796860_0001_1_01_000000_0
2016-01-14 17:16:37,128 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:37,128 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:37,129 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=ABA83F29D3C94F2F83BD33F06F37CF83, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:37,129 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791796860_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:37,129 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:37,129 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:37,130 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: ABA83F29D3C94F2F83BD33F06F37CF83
2016-01-14 17:16:37,131 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 678061632, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:37,131 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:37,131 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 678061632, type: INPUT, componentVertexName: ABA83F29D3C94F2F83BD33F06F37CF83
2016-01-14 17:16:37,132 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: ABA83F29D3C94F2F83BD33F06F37CF83
2016-01-14 17:16:37,132 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:37,133 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=0B25CD57BF164404AAE15C99804F5A98, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:37,133 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:37,134 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 0B25CD57BF164404AAE15C99804F5A98
2016-01-14 17:16:37,134 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:37,134 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 0B25CD57BF164404AAE15C99804F5A98
2016-01-14 17:16:37,138 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A620C68CD0354BF5B589522118A6B831, ordinal: 1
2016-01-14 17:16:37,138 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A620C68CD0354BF5B589522118A6B831, mem on start (mb), free: 369, total: 718, max: 718
2016-01-14 17:16:37,139 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:37,139 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:37,139 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:37,144 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:37,145 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 0B25CD57BF164404AAE15C99804F5A98
2016-01-14 17:16:37,145 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:37,145 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:37,145 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:37,145 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:37,145 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:37,145 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:37,145 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 678061632, TotalRequestedScaled: 6.25903044923077E8, TotalJVMHeap: 753401856, TotalAvailable: 504779243, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:37,146 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, ABA83F29D3C94F2F83BD33F06F37CF83, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=678061632, allocated=504779243
2016-01-14 17:16:37,146 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 0B25CD57BF164404AAE15C99804F5A98, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:37,146 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:37,146 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: ABA83F29D3C94F2F83BD33F06F37CF83 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:37,146 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: ABA83F29D3C94F2F83BD33F06F37CF83
2016-01-14 17:16:37,146 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:37,146 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:37,146 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:37,147 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:37,147 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: ABA83F29D3C94F2F83BD33F06F37CF83 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:37,147 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=678061632, postMergeMem=0, RuntimeTotalAvailable=504779243. Updated to: ShuffleMem=504779243, postMergeMem: 0
2016-01-14 17:16:37,147 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=504779243, maxSingleShuffleLimit=126194808, mergeThreshold=454301312, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:37,147 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:37,149 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: ABA83F29D3C94F2F83BD33F06F37CF83
2016-01-14 17:16:37,149 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:37,150 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791796860_0001_1_01_000000_0
2016-01-14 17:16:37,156 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:37,156 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791796860_0001_1_00_000000_0_10159, runDuration: 52260, ]
2016-01-14 17:16:37,156 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 7F44C4AD18454752916DC8FC0F06C50B
2016-01-14 17:16:37,156 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_ALL"], id: 0B25CD57BF164404AAE15C99804F5A98
2016-01-14 17:16:37,156 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_ALL"]
2016-01-14 17:16:37,157 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 7F44C4AD18454752916DC8FC0F06C50B, for 1 inputs
2016-01-14 17:16:37,159 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791796860_0001_1_00_000000_0_10159] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791797159, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 17:16:37,159 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : ABA83F29D3C94F2F83BD33F06F37CF83
2016-01-14 17:16:37,159 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:37,160 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [ABA83F29D3C94F2F83BD33F06F37CF83] #27 in 2ms
2016-01-14 17:16:37,160 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [ABA83F29D3C94F2F83BD33F06F37CF83] #27, status:false, isInterrupted:false
2016-01-14 17:16:37,161 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:37,161 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:37,161 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:37,161 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:37,162 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:37,162 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : ABA83F29D3C94F2F83BD33F06F37CF83
2016-01-14 17:16:37,162 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:37,162 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A620C68CD0354BF5B589522118A6B831, all 1 inputs ready in: 00:00:00.005
2016-01-14 17:16:37,163 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@5ff76ed3; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@6d8d0fdd
2016-01-14 17:16:37,167 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:37,167 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:37,167 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791796860_0001_1_01_000000_0
2016-01-14 17:16:37,167 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791796860_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:37,168 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791796860_0001_1_01_000000_0
2016-01-14 17:16:37,168 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A620C68CD0354BF5B589522118A6B831, mem on close (mb), free: 366, total: 718, max: 718
2016-01-14 17:16:37,168 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: ABA83F29D3C94F2F83BD33F06F37CF83
2016-01-14 17:16:37,170 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:37,170 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:37,170 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:37,171 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:37,171 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:37 Completed running task attempt: attempt_1452791796860_0001_1_01_000000_0
2016-01-14 17:16:37,172 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791796860_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:37,173 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791796860_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A620C68CD0354BF5B589522118A6B831, taskAttemptId=attempt_1452791796860_0001_1_01_000000_0, startTime=1452791797124, finishTime=1452791797173, timeTaken=49, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=753401856, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=13, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=11, LAST_EVENT_RECEIVED=11, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791797162, Process_Duration=6, Process_End_Time=1452791797168, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:37,173 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791796860_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:37,173 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791796860_0001_1_01_000000_0
2016-01-14 17:16:37,174 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:37,174 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791796860_0001_1][Event:TASK_FINISHED]: vertexName=A620C68CD0354BF5B589522118A6B831, taskId=task_1452791796860_0001_1_01_000000, startTime=1452791797124, finishTime=1452791797173, timeTaken=49, status=SUCCEEDED, successfulAttemptID=attempt_1452791796860_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=753401856, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=13, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=11, LAST_EVENT_RECEIVED=11, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791797162, Process_Duration=6, Process_End_Time=1452791797168, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:37,174 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791796860_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:37,174 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:37,174 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791796860_0001_1_01 [A620C68CD0354BF5B589522118A6B831] : 1
2016-01-14 17:16:37,174 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791796860_0001_1_01 [A620C68CD0354BF5B589522118A6B831]
2016-01-14 17:16:37,175 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:37,175 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:37,175 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791796860_0001_1][Event:VERTEX_FINISHED]: vertexName=A620C68CD0354BF5B589522118A6B831, vertexId=vertex_1452791796860_0001_1_01, initRequestedTime=1452791797030, initedTime=1452791797037, startRequestedTime=1452791797041, startedTime=1452791797041, finishTime=1452791797174, timeTaken=133, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=753401856, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=13, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=11, LAST_EVENT_RECEIVED=11, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791797162, Process_Duration=6, Process_End_Time=1452791797168, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452791797124, firstTasksToStart=[ task_1452791796860_0001_1_01_000000 ], lastTaskFinishTime=1452791797173, lastTasksToFinish=[ task_1452791796860_0001_1_01_000000 ], minTaskDuration=49, maxTaskDuration=49, avgTaskDuration=49.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791796860_0001_1_01_000000 ], longestDurationTasks=[ task_1452791796860_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:37,175 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791796860_0001_1_01 [A620C68CD0354BF5B589522118A6B831] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:37,176 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791796860_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:37,176 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791796860_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791796860_0001_00_000002, stoppedTime=1452791797176, exitStatus=0
2016-01-14 17:16:37,176 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:37,176 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791796860_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:37,176 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791796860_0001_1_01 [A620C68CD0354BF5B589522118A6B831] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:37,176 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791796860_0001_1
2016-01-14 17:16:37,176 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791796860_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452791796860_0001_1
2016-01-14 17:16:37,177 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452791796860_0001_1_00 [ABA83F29D3C94F2F83BD33F06F37CF83]
2016-01-14 17:16:37,177 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 0B25CD57BF164404AAE15C99804F5A98 for vertex: vertex_1452791796860_0001_1_01 [A620C68CD0354BF5B589522118A6B831]
2016-01-14 17:16:37,181 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=753401856, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=13, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=11, LAST_EVENT_RECEIVED=11][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791797162, Process_Duration=6, Process_End_Time=1452791797168, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 17:16:37,181 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:37,181 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:37,181 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791796860_0001_00_000002
2016-01-14 17:16:37,182 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791796860_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 17:16:37,182 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791796860_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:37,182 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791796860_0001_00_000002, Exiting
2016-01-14 17:16:37,182 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791796860_0001_00_000002
2016-01-14 17:16:37,182 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791796860_0001_00_000002
2016-01-14 17:16:37,183 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791796860_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:37,183 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791796860_0001_00_000002 completed successfully
2016-01-14 17:16:37,190 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791796860_0001_1, queueSize=0
2016-01-14 17:16:37,190 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:37,190 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791796860_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791796860_0001_1, startTime=1452791797028, finishTime=1452791797188, timeTaken=160, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=571, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=17, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1506803712, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=13, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=11, LAST_EVENT_RECEIVED=11, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905583594265, Process_Duration=8, Process_End_Time=2905583594273, Read_Duration=3, Tuples_Read=40, Tuples_Written=40, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 17:16:37,190 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791796860_0001_1 finished with state: SUCCEEDED
2016-01-14 17:16:37,190 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791796860_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:37,191 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791796860_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:16:37 Completed Dag: dag_1452791796860_0001_1
2016-01-14 17:16:37,191 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452791796860_0001_00_000002
2016-01-14 17:16:37,191 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791796860_0001_1, dagState=SUCCEEDED
2016-01-14 17:16:37,191 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791796860_0001_1_post
2016-01-14 17:16:37,191 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:37,191 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:37,197 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...blyplatform/0_UNKNOWN_ALL, applicationId=application_1452791796860_0001
2016-01-14 17:16:37,199 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:37,199 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:37,199 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:37,199 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:37,199 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:37,199 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:37,200 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:37,200 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:37,200 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:37,200 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:37,200 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:37,200 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:37,200 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:37,205 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/53D6E9FE022F4473909CEBB0DC81DF55/C676A7E2584F4A6D86059FB08537B53C/.tez/application_1452791796860_0001
2016-01-14 17:16:37,206 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.370
2016-01-14 17:16:37,270 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_UNKNOWN_ALL
2016-01-14 17:16:37,291 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:37,291 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:37,321 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.007
2016-01-14 17:16:37,327 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-01-14 17:16:37,328 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:37,328 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:37,329 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:37,342 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:37,347 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_REPLACE
2016-01-14 17:16:37,389 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:37,390 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:37,390 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_REPLACE"]
2016-01-14 17:16:37,390 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:37,390 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:37,390 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:37,390 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...latform/0_UNKNOWN_REPLACE
2016-01-14 17:16:37,392 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:37,403 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:37,408 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:37,408 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:37,409 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:37,418 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3A8CB90EF76D4599B72BB1C641C4D95F/B0891A0285B2493181210F9255622E5C/.tez/application_1452791797408_0001 doesn't exist and is created
2016-01-14 17:16:37,433 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:37,433 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:37,434 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3A8CB90EF76D4599B72BB1C641C4D95F/B0891A0285B2493181210F9255622E5C/.tez/application_1452791797408_0001
2016-01-14 17:16:37,436 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791797408_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:37,447 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:37,447 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:37,449 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:37,449 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:37,449 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3A8CB90EF76D4599B72BB1C641C4D95F/B0891A0285B2493181210F9255622E5C/.tez/application_1452791797408_0001/localmode-log-dir
2016-01-14 17:16:37,449 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3A8CB90EF76D4599B72BB1C641C4D95F/B0891A0285B2493181210F9255622E5C/.tez/application_1452791797408_0001/localmode-log-dir/history.txt.appattempt_1452791797408_0001_000000, maxErrors=10
2016-01-14 17:16:37,449 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:37,449 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791797408_0001_000000, appSubmitTime=1452791797436, launchTime=1452791797436
2016-01-14 17:16:37,453 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:37,454 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:37,455 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:37,457 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:37,457 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:40708
2016-01-14 17:16:37,460 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:37,461 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791797408_0001_000000, startTime=1452791797461
2016-01-14 17:16:37,461 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:37,462 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:37,533 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:37,533 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:37,534 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/0_UNKNOWN_REPLACE, applicationId=application_1452791797408_0001, dagName=[3A8CB90EF76D4599B72BB1C641C4D95F/B0891A0285B2493181210F9255622E5C] every/(1/1) ...latform/0_UNKNOWN_REPLACE
2016-01-14 17:16:37,548 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [3A8CB90EF76D4599B72BB1C641C4D95F/B0891A0285B2493181210F9255622E5C] every/(1/1) ...latform/0_UNKNOWN_REPLACE
2016-01-14 17:16:37,549 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791797408_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3A8CB90EF76D4599B72BB1C641C4D95F/B0891A0285B2493181210F9255622E5C/.tez/application_1452791797408_0001/localmode-log-dir/dag_1452791797408_0001_1.dot
2016-01-14 17:16:37,549 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3A8CB90EF76D4599B72BB1C641C4D95F/B0891A0285B2493181210F9255622E5C/.tez/application_1452791797408_0001/localmode-log-dir/dag_1452791797408_0001_1-tez-dag.pb.txt
2016-01-14 17:16:37,555 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791797408_0001_1
2016-01-14 17:16:37,555 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:37,555 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [3A8CB90EF76D4599B72BB1C641C4D95F/B0891A0285B2493181210F9255622E5C] every/(1/1) ...latform/0_UNKNOWN_REPLACE
2016-01-14 17:16:37 Running Dag: dag_1452791797408_0001_1
2016-01-14 17:16:37,567 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791797408_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791797408_0001_1, submitTime=1452791797548
2016-01-14 17:16:37,567 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:37,568 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex F2D7C7FB97B44BB09FE6BEE167706114
2016-01-14 17:16:37,568 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:37,568 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791797408_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791797408_0001_1, initTime=1452791797567
2016-01-14 17:16:37,568 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791797408_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:37,568 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:37,568 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791797408_0001_1][Event:DAG_STARTED]: dagID=dag_1452791797408_0001_1, startTime=1452791797568
2016-01-14 17:16:37,568 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:37,568 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791797408_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:37,568 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:37,569 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: E80FAB8B97DF4EE89B14962F8398D7DB : {71222A1731364AA186291DDD55C21D97={InputName=71222A1731364AA186291DDD55C21D97}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:37,569 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 71222A1731364AA186291DDD55C21D97, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:37,569 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791797408_0001_1_00 [E80FAB8B97DF4EE89B14962F8398D7DB]
2016-01-14 17:16:37,569 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791797408_0001_1_00 [E80FAB8B97DF4EE89B14962F8398D7DB]
2016-01-14 17:16:37,569 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791797408_0001_1_00 [E80FAB8B97DF4EE89B14962F8398D7DB]
2016-01-14 17:16:37,569 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791797408_0001_1_00 [E80FAB8B97DF4EE89B14962F8398D7DB]. Starting root input initializers: 1
2016-01-14 17:16:37,570 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791797408_0001_1_00 [E80FAB8B97DF4EE89B14962F8398D7DB] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:37,570 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 71222A1731364AA186291DDD55C21D97 on vertex vertex_1452791797408_0001_1_00 [E80FAB8B97DF4EE89B14962F8398D7DB]
2016-01-14 17:16:37,570 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791797408_0001_1_01 [F2D7C7FB97B44BB09FE6BEE167706114]
2016-01-14 17:16:37,572 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:37,572 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791797408_0001_1_01 [F2D7C7FB97B44BB09FE6BEE167706114]
2016-01-14 17:16:37,573 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791797408_0001_1_01 [F2D7C7FB97B44BB09FE6BEE167706114]
2016-01-14 17:16:37,573 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791797408_0001_1_01 [F2D7C7FB97B44BB09FE6BEE167706114]
2016-01-14 17:16:37,573 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=930D71B0D5F2490393EFDB587DE3E364, vertexId=vertex_1452791797408_0001_1_01 [F2D7C7FB97B44BB09FE6BEE167706114], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:37,573 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:37,573 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 71222A1731364AA186291DDD55C21D97 asking for 1 tasks. Headroom: 718 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:37,573 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=930D71B0D5F2490393EFDB587DE3E364, vertexId=vertex_1452791797408_0001_1_01 [F2D7C7FB97B44BB09FE6BEE167706114]
2016-01-14 17:16:37,573 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/0_UNKNOWN_REPLACE, applicationId=application_1452791797408_0001, dagName=[3A8CB90EF76D4599B72BB1C641C4D95F/B0891A0285B2493181210F9255622E5C] every/(1/1) ...latform/0_UNKNOWN_REPLACE
2016-01-14 17:16:37,574 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:37,574 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:37,575 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=930D71B0D5F2490393EFDB587DE3E364, vertexName=F2D7C7FB97B44BB09FE6BEE167706114, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:37,575 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=930D71B0D5F2490393EFDB587DE3E364, vertexId=vertex_1452791797408_0001_1_01 [F2D7C7FB97B44BB09FE6BEE167706114]
2016-01-14 17:16:37,575 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:37,576 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 71222A1731364AA186291DDD55C21D97 on vertex vertex_1452791797408_0001_1_00 [E80FAB8B97DF4EE89B14962F8398D7DB]
2016-01-14 17:16:37,576 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:37,576 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791797408_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F2D7C7FB97B44BB09FE6BEE167706114, vertexId=vertex_1452791797408_0001_1_01, initRequestedTime=1452791797570, initedTime=1452791797576, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:37,576 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791797408_0001_1_01 [F2D7C7FB97B44BB09FE6BEE167706114] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:37,576 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791797408_0001_1_00 [E80FAB8B97DF4EE89B14962F8398D7DB] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:37,577 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {71222A1731364AA186291DDD55C21D97=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:37,577 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791797408_0001_1_00 [E80FAB8B97DF4EE89B14962F8398D7DB] parallelism set to 1
2016-01-14 17:16:37,577 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791797408_0001_1_00 [E80FAB8B97DF4EE89B14962F8398D7DB] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:37,577 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791797408_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791797408_0001_1_00, eventCount=1
2016-01-14 17:16:37,578 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:37,578 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791797408_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E80FAB8B97DF4EE89B14962F8398D7DB, vertexId=vertex_1452791797408_0001_1_00, initRequestedTime=1452791797569, initedTime=1452791797578, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:37,578 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791797408_0001_1_00 [E80FAB8B97DF4EE89B14962F8398D7DB] with distanceFromRoot: 0
2016-01-14 17:16:37,579 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791797408_0001_1_00 [E80FAB8B97DF4EE89B14962F8398D7DB] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:37,579 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in E80FAB8B97DF4EE89B14962F8398D7DB
2016-01-14 17:16:37,579 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791797408_0001_1_00 [E80FAB8B97DF4EE89B14962F8398D7DB]
2016-01-14 17:16:37,579 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:37,580 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791797408_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791797408_0001_1_00, startRequestedTime=1452791797579, startedTime=1452791797579
2016-01-14 17:16:37,580 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791797408_0001_1_00 [E80FAB8B97DF4EE89B14962F8398D7DB] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:37,580 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:37,580 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791797408_0001_1][Event:TASK_STARTED]: vertexName=E80FAB8B97DF4EE89B14962F8398D7DB, taskId=task_1452791797408_0001_1_00_000000, scheduledTime=1452791797580, launchTime=1452791797580
2016-01-14 17:16:37,581 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791797408_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:37,581 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791797408_0001_1_00 for vertex: vertex_1452791797408_0001_1_01 [F2D7C7FB97B44BB09FE6BEE167706114] numStartedSources: 1 numSources: 1
2016-01-14 17:16:37,581 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791797408_0001_1_01 [F2D7C7FB97B44BB09FE6BEE167706114] with distanceFromRoot: 1
2016-01-14 17:16:37,581 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791797408_0001_1_00 [E80FAB8B97DF4EE89B14962F8398D7DB]
2016-01-14 17:16:37,581 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: E80FAB8B97DF4EE89B14962F8398D7DB in vertex: F2D7C7FB97B44BB09FE6BEE167706114
2016-01-14 17:16:37,582 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: F2D7C7FB97B44BB09FE6BEE167706114 with 1 source tasks and 1 pending tasks
2016-01-14 17:16:37,582 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:37,582 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791797408_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791797408_0001_1_01, startRequestedTime=1452791797581, startedTime=1452791797581
2016-01-14 17:16:37,582 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791797408_0001_1_01 [F2D7C7FB97B44BB09FE6BEE167706114] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:37,583 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [3A8CB90EF76D4599B72BB1C641C4D95F/B0891A0285B2493181210F9255622E5C] every/(1/1) ...latform/0_UNKNOWN_REPLACE, VertexName: E80FAB8B97DF4EE89B14962F8398D7DB, VertexParallelism: 1, TaskAttemptID:attempt_1452791797408_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=71222A1731364AA186291DDD55C21D97, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F2D7C7FB97B44BB09FE6BEE167706114, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:37,583 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791797408_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:37,583 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:37,584 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:37,584 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:37,584 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:37,585 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791797408_0001_00_000001
2016-01-14 17:16:37,585 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791797408_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:37,586 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791797408_0001_00_000001
2016-01-14 17:16:37,586 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:37,587 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791797408_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:37,587 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791797408_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791797408_0001_00_000001, launchTime=1452791797587
2016-01-14 17:16:37,587 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:37,588 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 17:16:37,588 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 17:16:37,588 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791797408_0001_00_000001
2016-01-14 17:16:37,588 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791797408_0001_1_00_000000_0] to container: [container_1452791797408_0001_00_000001]
2016-01-14 17:16:37,588 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791797408_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:37,589 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791797408_0001_1_00_000000_0] started. Is using containerId: [container_1452791797408_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:37,589 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791797408_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E80FAB8B97DF4EE89B14962F8398D7DB, taskAttemptId=attempt_1452791797408_0001_1_00_000000_0, startTime=1452791797589, containerId=container_1452791797408_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791797408_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:37,589 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:37,589 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791797408_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:37,589 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791797408_0001, with dag id: dag_1452791797408_0001_1
2016-01-14 17:16:37,589 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791797408_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:37,590 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791797408_0001_00_000001 given task: attempt_1452791797408_0001_1_00_000000_0
2016-01-14 17:16:37,590 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791797408_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791797408_0001_1_00_000000_0
2016-01-14 17:16:37 Starting to run new task attempt: attempt_1452791797408_0001_1_00_000000_0
2016-01-14 17:16:37,590 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791797408_0001_1_00_000000_0
2016-01-14 17:16:37,590 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:37,590 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:37,590 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:37,590 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:37,591 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:37,591 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:37,591 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [3A8CB90EF76D4599B72BB1C641C4D95F/B0891A0285B2493181210F9255622E5C] every/(1/1) ...latform/0_UNKNOWN_REPLACE, VertexName: E80FAB8B97DF4EE89B14962F8398D7DB, VertexParallelism: 1, TaskAttemptID:attempt_1452791797408_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=71222A1731364AA186291DDD55C21D97, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F2D7C7FB97B44BB09FE6BEE167706114, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:37,591 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=753401856
2016-01-14 17:16:37,591 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791797408_0001_1_00_000000_0
2016-01-14 17:16:37,591 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:37,591 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:37,592 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791797408_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:37,593 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=71222A1731364AA186291DDD55C21D97, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:37,593 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:37,593 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 71222A1731364AA186291DDD55C21D97
2016-01-14 17:16:37,593 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 71222A1731364AA186291DDD55C21D97
2016-01-14 17:16:37,593 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:37,594 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F2D7C7FB97B44BB09FE6BEE167706114, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:37,595 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:37,595 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F2D7C7FB97B44BB09FE6BEE167706114
2016-01-14 17:16:37,596 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:37,596 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:37,596 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: F2D7C7FB97B44BB09FE6BEE167706114
2016-01-14 17:16:37,597 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F2D7C7FB97B44BB09FE6BEE167706114
2016-01-14 17:16:37,599 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:37,599 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 71222A1731364AA186291DDD55C21D97
2016-01-14 17:16:37,599 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 71222A1731364AA186291DDD55C21D97
2016-01-14 17:16:37,601 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E80FAB8B97DF4EE89B14962F8398D7DB, ordinal: 0
2016-01-14 17:16:37,601 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E80FAB8B97DF4EE89B14962F8398D7DB, mem on start (mb), free: 276, total: 718, max: 718
2016-01-14 17:16:37,602 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:37,602 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:37,602 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:37,602 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:37,602 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:37,602 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:37,602 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:37,602 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:37,602 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:37,603 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 753401856, TotalAvailable: 504779243, TotalRequested/TotalJVMHeap:0.14
2016-01-14 17:16:37,603 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F2D7C7FB97B44BB09FE6BEE167706114, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:37,603 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 71222A1731364AA186291DDD55C21D97, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:37,603 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:37,603 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 71222A1731364AA186291DDD55C21D97 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:37,603 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:37,603 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:37,603 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 71222A1731364AA186291DDD55C21D97
2016-01-14 17:16:37,603 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:37,604 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 71222A1731364AA186291DDD55C21D97
2016-01-14 17:16:37,604 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:37,604 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791797408_0001_1_00_000000_0
2016-01-14 17:16:37,604 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:37,604 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@d5ee11a
2016-01-14 17:16:37,607 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:37,607 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@d5ee11a
2016-01-14 17:16:37,607 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:37,608 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:37,608 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 71222A1731364AA186291DDD55C21D97
2016-01-14 17:16:37,608 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 72B216045329432CB43A2EB002454EE4
2016-01-14 17:16:37,608 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 72B216045329432CB43A2EB002454EE4
2016-01-14 17:16:37,609 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@70ccbbea; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@11453981; comparator=cascading.tuple.hadoop.util.TupleComparator@57c24d02; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:37,609 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:37,609 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:37,609 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:37,643 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:37,643 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:37,643 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:37,644 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:37,644 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:37,645 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:37,646 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:37,646 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E80FAB8B97DF4EE89B14962F8398D7DB, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:37,649 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791797408_0001_1_00_000000_0
2016-01-14 17:16:37,649 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E80FAB8B97DF4EE89B14962F8398D7DB, mem on close (mb), free: 371, total: 714, max: 714
2016-01-14 17:16:37,649 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:37,649 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:37,649 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:37,649 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:37,651 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:37 Completed running task attempt: attempt_1452791797408_0001_1_00_000000_0
2016-01-14 17:16:37,654 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791797408_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:37,654 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=21, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=749207552, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791797646, Process_Duration=3, Process_End_Time=1452791797649, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:37,654 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791797408_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791797408_0001_1_00, eventCount=1
2016-01-14 17:16:37,655 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:37,655 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:37,655 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791797408_0001_00_000001
2016-01-14 17:16:37,655 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:37,655 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791797408_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E80FAB8B97DF4EE89B14962F8398D7DB, taskAttemptId=attempt_1452791797408_0001_1_00_000000_0, startTime=1452791797589, finishTime=1452791797655, timeTaken=66, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=21, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=749207552, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791797646, Process_Duration=3, Process_End_Time=1452791797649, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:37,655 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791797408_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:37,655 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:37,655 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791797408_0001_1_00_000000_0
2016-01-14 17:16:37,657 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:37,657 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791797408_0001_1][Event:TASK_FINISHED]: vertexName=E80FAB8B97DF4EE89B14962F8398D7DB, taskId=task_1452791797408_0001_1_00_000000, startTime=1452791797589, finishTime=1452791797656, timeTaken=67, status=SUCCEEDED, successfulAttemptID=attempt_1452791797408_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=21, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=749207552, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791797646, Process_Duration=3, Process_End_Time=1452791797649, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:37,657 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791797408_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:37,657 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791797408_0001_1_00 [E80FAB8B97DF4EE89B14962F8398D7DB] : 1
2016-01-14 17:16:37,657 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791797408_0001_1_00 [E80FAB8B97DF4EE89B14962F8398D7DB]
2016-01-14 17:16:37,658 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791797408_0001_1][Event:VERTEX_FINISHED]: vertexName=E80FAB8B97DF4EE89B14962F8398D7DB, vertexId=vertex_1452791797408_0001_1_00, initRequestedTime=1452791797569, initedTime=1452791797578, startRequestedTime=1452791797579, startedTime=1452791797579, finishTime=1452791797657, timeTaken=78, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=21, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=749207552, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791797646, Process_Duration=3, Process_End_Time=1452791797649, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791797589, firstTasksToStart=[ task_1452791797408_0001_1_00_000000 ], lastTaskFinishTime=1452791797655, lastTasksToFinish=[ task_1452791797408_0001_1_00_000000 ], minTaskDuration=66, maxTaskDuration=66, avgTaskDuration=66.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791797408_0001_1_00_000000 ], longestDurationTasks=[ task_1452791797408_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:37,658 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:37,658 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791797408_0001_1_00 [E80FAB8B97DF4EE89B14962F8398D7DB] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:37,658 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791797408_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:37,659 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791797408_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791797408_0001_00_000001, stoppedTime=1452791797659, exitStatus=0
2016-01-14 17:16:37,659 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791797408_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:37,659 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:37,659 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791797408_0001_1_01 [F2D7C7FB97B44BB09FE6BEE167706114] attempt: attempt_1452791797408_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:37,659 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: F2D7C7FB97B44BB09FE6BEE167706114
2016-01-14 17:16:37,659 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791797408_0001_1_01 [F2D7C7FB97B44BB09FE6BEE167706114]
2016-01-14 17:16:37,659 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791797408_0001_1_00 [E80FAB8B97DF4EE89B14962F8398D7DB] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:37,659 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791797408_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:37,660 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791797408_0001_1][Event:TASK_STARTED]: vertexName=F2D7C7FB97B44BB09FE6BEE167706114, taskId=task_1452791797408_0001_1_01_000000, scheduledTime=1452791797660, launchTime=1452791797660
2016-01-14 17:16:37,660 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791797408_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:37,660 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:37,660 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791797408_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:37,660 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791797408_0001_1_01 [F2D7C7FB97B44BB09FE6BEE167706114]
2016-01-14 17:16:37,661 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [3A8CB90EF76D4599B72BB1C641C4D95F/B0891A0285B2493181210F9255622E5C] every/(1/1) ...latform/0_UNKNOWN_REPLACE, VertexName: F2D7C7FB97B44BB09FE6BEE167706114, VertexParallelism: 1, TaskAttemptID:attempt_1452791797408_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E80FAB8B97DF4EE89B14962F8398D7DB, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=930D71B0D5F2490393EFDB587DE3E364, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:37,661 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791797408_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:37,661 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:37,661 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791797408_0001_00_000002
2016-01-14 17:16:37,662 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791797408_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:37,663 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791797408_0001_00_000002
2016-01-14 17:16:37,663 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:37,664 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791797408_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:37,664 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791797408_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791797408_0001_00_000002, launchTime=1452791797664
2016-01-14 17:16:37,664 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:37,665 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791797408_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:37,665 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791797408_0001_00_000002
2016-01-14 17:16:37,665 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791797408_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:37,665 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791797408_0001_00_000001, Exiting
2016-01-14 17:16:37,665 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791797408_0001_00_000001
2016-01-14 17:16:37,666 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791797408_0001_00_000001
2016-01-14 17:16:37,666 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791797408_0001_1_01_000000_0] to container: [container_1452791797408_0001_00_000002]
2016-01-14 17:16:37,666 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791797408_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:37,666 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791797408_0001_1_01_000000_0] started. Is using containerId: [container_1452791797408_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:37,667 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791797408_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F2D7C7FB97B44BB09FE6BEE167706114, taskAttemptId=attempt_1452791797408_0001_1_01_000000_0, startTime=1452791797666, containerId=container_1452791797408_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791797408_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:37,667 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791797408_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:37,667 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791797408_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:37,667 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:37,667 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791797408_0001_00_000002 given task: attempt_1452791797408_0001_1_01_000000_0
2016-01-14 17:16:37,667 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791797408_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791797408_0001_1_01_000000_0
2016-01-14 17:16:37,667 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791797408_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:37 Starting to run new task attempt: attempt_1452791797408_0001_1_01_000000_0
2016-01-14 17:16:37,668 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791797408_0001_1_01_000000_0
2016-01-14 17:16:37,668 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:37,668 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:37,668 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:37,668 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:37,668 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:37,668 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791797408_0001_00_000001 completed successfully
2016-01-14 17:16:37,668 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:37,669 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [3A8CB90EF76D4599B72BB1C641C4D95F/B0891A0285B2493181210F9255622E5C] every/(1/1) ...latform/0_UNKNOWN_REPLACE, VertexName: F2D7C7FB97B44BB09FE6BEE167706114, VertexParallelism: 1, TaskAttemptID:attempt_1452791797408_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E80FAB8B97DF4EE89B14962F8398D7DB, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=930D71B0D5F2490393EFDB587DE3E364, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:37,669 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=749207552
2016-01-14 17:16:37,669 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791797408_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:37,669 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791797408_0001_1_01_000000_0
2016-01-14 17:16:37,669 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:37,669 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:37,670 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:37,670 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=E80FAB8B97DF4EE89B14962F8398D7DB, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:37,670 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791797408_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:37,671 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:37,671 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: E80FAB8B97DF4EE89B14962F8398D7DB
2016-01-14 17:16:37,673 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:37,673 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 674286784, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:37,673 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:37,673 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 674286784, type: INPUT, componentVertexName: E80FAB8B97DF4EE89B14962F8398D7DB
2016-01-14 17:16:37,673 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: E80FAB8B97DF4EE89B14962F8398D7DB
2016-01-14 17:16:37,671 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=930D71B0D5F2490393EFDB587DE3E364, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:37,674 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:37,674 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 930D71B0D5F2490393EFDB587DE3E364
2016-01-14 17:16:37,674 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:37,675 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 930D71B0D5F2490393EFDB587DE3E364
2016-01-14 17:16:37,679 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F2D7C7FB97B44BB09FE6BEE167706114, ordinal: 1
2016-01-14 17:16:37,679 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F2D7C7FB97B44BB09FE6BEE167706114, mem on start (mb), free: 366, total: 714, max: 714
2016-01-14 17:16:37,679 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:37,679 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:37,679 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:37,685 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:37,686 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 930D71B0D5F2490393EFDB587DE3E364
2016-01-14 17:16:37,686 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:37,686 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:37,686 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:37,686 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:37,686 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:37,686 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:37,686 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 674286784, TotalRequestedScaled: 6.224185698461539E8, TotalJVMHeap: 749207552, TotalAvailable: 501969059, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:37,686 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, E80FAB8B97DF4EE89B14962F8398D7DB, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=674286784, allocated=501969059
2016-01-14 17:16:37,687 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 930D71B0D5F2490393EFDB587DE3E364, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:37,687 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:37,687 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: E80FAB8B97DF4EE89B14962F8398D7DB being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:37,687 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:37,687 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:37,687 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: E80FAB8B97DF4EE89B14962F8398D7DB
2016-01-14 17:16:37,688 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:37,688 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:37,688 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: E80FAB8B97DF4EE89B14962F8398D7DB with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:37,689 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=674286784, postMergeMem=0, RuntimeTotalAvailable=501969059. Updated to: ShuffleMem=501969059, postMergeMem: 0
2016-01-14 17:16:37,689 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=501969059, maxSingleShuffleLimit=125492264, mergeThreshold=451772128, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:37,689 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:37,691 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: E80FAB8B97DF4EE89B14962F8398D7DB
2016-01-14 17:16:37,692 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:37,693 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791797408_0001_1_01_000000_0
2016-01-14 17:16:37,695 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791797408_0001_1_00_000000_0_10165, runDuration: 57809, ]
2016-01-14 17:16:37,696 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791797408_0001_1_00_000000_0_10165] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791797695, TimeTaken=0, Rate=0.00 MB/s
2016-01-14 17:16:37,696 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : E80FAB8B97DF4EE89B14962F8398D7DB
2016-01-14 17:16:37,696 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:37,696 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 72B216045329432CB43A2EB002454EE4
2016-01-14 17:16:37,697 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_REPLACE"], id: 930D71B0D5F2490393EFDB587DE3E364
2016-01-14 17:16:37,697 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_REPLACE"]
2016-01-14 17:16:37,697 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 72B216045329432CB43A2EB002454EE4, for 1 inputs
2016-01-14 17:16:37,697 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:37,697 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [E80FAB8B97DF4EE89B14962F8398D7DB] #28 in 2ms
2016-01-14 17:16:37,698 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [E80FAB8B97DF4EE89B14962F8398D7DB] #28, status:false, isInterrupted:false
2016-01-14 17:16:37,698 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:37,698 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:37,699 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:37,699 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:37,699 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:37,699 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : E80FAB8B97DF4EE89B14962F8398D7DB
2016-01-14 17:16:37,699 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F2D7C7FB97B44BB09FE6BEE167706114, all 1 inputs ready in: 00:00:00.002
2016-01-14 17:16:37,699 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:37,700 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@4c6e6341; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@6336679a
2016-01-14 17:16:37,701 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:37,701 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(92)) - caught throwable
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:37,701 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:37,701 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:37,702 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791797408_0001_1_01_000000_0
2016-01-14 17:16:37,702 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791797408_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:37,702 INFO  task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1452791797408_0001_1_01_000000_0
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:37,703 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=749207552, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791797699, Process_Duration=3, Process_End_Time=1452791797702, Read_Duration=1, Tuples_Read=1]]
2016-01-14 17:16:37,703 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F2D7C7FB97B44BB09FE6BEE167706114, mem on close (mb), free: 362, total: 714, max: 714
2016-01-14 17:16:37,703 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=F2D7C7FB97B44BB09FE6BEE167706114, index=1
2016-01-14 17:16:37,704 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: E80FAB8B97DF4EE89B14962F8398D7DB
2016-01-14 17:16:37,704 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:37,704 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:37,704 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:37,704 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:37,704 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:37,704 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=F2D7C7FB97B44BB09FE6BEE167706114, sourceVertex=E80FAB8B97DF4EE89B14962F8398D7DB
2016-01-14 17:16:37,705 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=F2D7C7FB97B44BB09FE6BEE167706114, sourceVertex=930D71B0D5F2490393EFDB587DE3E364
2016-01-14 17:16:37,705 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791797408_0001_00_000002
2016-01-14 17:16:37,705 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791797408_0001_00_000002
2016-01-14 17:16:37,705 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791797408_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-01-14 17:16:37,705 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1452791797408_0001_00_000002 completed but with errors
2016-01-14 17:16:37,708 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(686)) - Container container_1452791797408_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-01-14 17:16:37,708 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791797408_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791797408_0001_00_000002, stoppedTime=1452791797708, exitStatus=1
2016-01-14 17:16:37,708 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791797408_0001_00_000002 transitioned from RUNNING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:37,709 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791797408_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F2D7C7FB97B44BB09FE6BEE167706114, taskAttemptId=attempt_1452791797408_0001_1_01_000000_0, startTime=1452791797666, finishTime=1452791797708, timeTaken=42, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=2, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=749207552, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791797699, Process_Duration=3, Process_End_Time=1452791797702, Read_Duration=1, Tuples_Read=1
2016-01-14 17:16:37,709 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791797408_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-01-14 17:16:37,709 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791797408_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-01-14 17:16:37,709 INFO  impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1452791797408_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-01-14 17:16:37,709 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791797408_0001_1][Event:TASK_FINISHED]: vertexName=F2D7C7FB97B44BB09FE6BEE167706114, taskId=task_1452791797408_0001_1_01_000000, startTime=1452791797666, finishTime=1452791797709, timeTaken=43, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
, Container container_1452791797408_0001_00_000002 finished with diagnostics set to [TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException]], counters=Counters: 0
2016-01-14 17:16:37,709 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791797408_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-01-14 17:16:37,709 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:37,709 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791797408_0001_1_01 [F2D7C7FB97B44BB09FE6BEE167706114] : 1
2016-01-14 17:16:37,710 INFO  impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1452791797408_0001_1_01 [F2D7C7FB97B44BB09FE6BEE167706114] because task failed: task_1452791797408_0001_1_01_000000
2016-01-14 17:16:37,710 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:37,710 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:37,710 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:37,710 INFO  impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1452791797408_0001_1_01 [F2D7C7FB97B44BB09FE6BEE167706114] due to trigger: OWN_TASK_FAILURE
2016-01-14 17:16:37,711 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-01-14 17:16:37,711 INFO  impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1452791797408_0001_1_01 [F2D7C7FB97B44BB09FE6BEE167706114]
2016-01-14 17:16:37,711 INFO  impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=930D71B0D5F2490393EFDB587DE3E364, vertexId=vertex_1452791797408_0001_1_01 [F2D7C7FB97B44BB09FE6BEE167706114]
2016-01-14 17:16:37,712 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791797408_0001_1][Event:VERTEX_FINISHED]: vertexName=F2D7C7FB97B44BB09FE6BEE167706114, vertexId=vertex_1452791797408_0001_1_01, initRequestedTime=1452791797570, initedTime=1452791797576, startRequestedTime=1452791797581, startedTime=1452791797581, finishTime=1452791797712, timeTaken=131, status=FAILED, diagnostics=Task failed, taskId=task_1452791797408_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
, Container container_1452791797408_0001_00_000002 finished with diagnostics set to [TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException]]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1452791797408_0001_1_01 [F2D7C7FB97B44BB09FE6BEE167706114] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1452791797666, firstTasksToStart=[ task_1452791797408_0001_1_01_000000 ], lastTaskFinishTime=1452791797708, lastTasksToFinish=[ task_1452791797408_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[  ], longestDurationTasks=[  ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-01-14 17:16:37,713 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791797408_0001_1_01 [F2D7C7FB97B44BB09FE6BEE167706114] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-01-14 17:16:37,713 INFO  node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1452791797408_0001_1_01_000000_0 failed: true container: container_1452791797408_0001_00_000002 numFailedTAs: 0
2016-01-14 17:16:37,713 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791797408_0001_1_01 [F2D7C7FB97B44BB09FE6BEE167706114] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-01-14 17:16:37,713 INFO  impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-01-14 17:16:37,713 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791797408_0001_1
2016-01-14 17:16:37,713 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 930D71B0D5F2490393EFDB587DE3E364 for vertex: vertex_1452791797408_0001_1_01 [F2D7C7FB97B44BB09FE6BEE167706114]
2016-01-14 17:16:37,713 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:37,713 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1452791797408_0001_1_00 [E80FAB8B97DF4EE89B14962F8398D7DB]
2016-01-14 17:16:37,715 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791797408_0001_1, queueSize=0
2016-01-14 17:16:37,715 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:37,716 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791797408_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791797408_0001_1, startTime=1452791797568, finishTime=1452791797714, timeTaken=146, status=FAILED, diagnostics=Vertex failed, vertexName=F2D7C7FB97B44BB09FE6BEE167706114, vertexId=vertex_1452791797408_0001_1_01, diagnostics=[Task failed, taskId=task_1452791797408_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
, Container container_1452791797408_0001_00_000002 finished with diagnostics set to [TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException]]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1452791797408_0001_1_01 [F2D7C7FB97B44BB09FE6BEE167706114] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=21, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=749207552, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791797646, Process_Duration=3, Process_End_Time=1452791797649, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:37,716 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791797408_0001_1 finished with state: FAILED
2016-01-14 17:16:37,716 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791797408_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:37 Completed Dag: dag_1452791797408_0001_1
2016-01-14 17:16:37,716 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791797408_0001_1, dagState=FAILED
2016-01-14 17:16:37,716 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791797408_0001_1_post
2016-01-14 17:16:37,717 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:37,717 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:37,733 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/0_UNKNOWN_REPLACE, applicationId=application_1452791797408_0001
2016-01-14 17:16:37,735 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:37,735 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:37,735 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:37,735 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:37,735 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:37,736 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:37,736 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:37,737 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:37,737 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:37,737 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:37,737 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:37,737 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:37,739 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:37,742 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3A8CB90EF76D4599B72BB1C641C4D95F/B0891A0285B2493181210F9255622E5C/.tez/application_1452791797408_0001
2016-01-14 17:16:37,742 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping all jobs
2016-01-14 17:16:37,742 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping: (1/1) ...latform/0_UNKNOWN_REPLACE
2016-01-14 17:16:37,743 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopped all jobs
2016-01-14 17:16:37,743 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.354
2016-01-14 17:16:37,743 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_UNKNOWN_REPLACE
2016-01-14 17:16:37,775 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:37,775 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:37,812 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.006
2016-01-14 17:16:37,821 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-01-14 17:16:37,821 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:37,821 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:37,822 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:37,834 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:37,839 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_SWAP
2016-01-14 17:16:37,879 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:37,879 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:37,879 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_SWAP"]
2016-01-14 17:16:37,879 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:37,880 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:37,880 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:37,881 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...lyplatform/0_UNKNOWN_SWAP
2016-01-14 17:16:37,882 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:37,893 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:37,897 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:37,898 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:37,898 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:37,906 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E4B14FA96F2E491FBD6D0B46F61504BF/F527F4C49E52423FA3C5013B32FC4EF5/.tez/application_1452791797897_0001 doesn't exist and is created
2016-01-14 17:16:37,923 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:37,923 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:37,923 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E4B14FA96F2E491FBD6D0B46F61504BF/F527F4C49E52423FA3C5013B32FC4EF5/.tez/application_1452791797897_0001
2016-01-14 17:16:37,926 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791797897_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:37,935 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:37,936 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:37,937 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:37,937 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:37,937 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E4B14FA96F2E491FBD6D0B46F61504BF/F527F4C49E52423FA3C5013B32FC4EF5/.tez/application_1452791797897_0001/localmode-log-dir
2016-01-14 17:16:37,937 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E4B14FA96F2E491FBD6D0B46F61504BF/F527F4C49E52423FA3C5013B32FC4EF5/.tez/application_1452791797897_0001/localmode-log-dir/history.txt.appattempt_1452791797897_0001_000000, maxErrors=10
2016-01-14 17:16:37,937 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:37,937 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791797897_0001_000000, appSubmitTime=1452791797925, launchTime=1452791797925
2016-01-14 17:16:37,939 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:37,941 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:37,944 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:37,945 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:37,947 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:44385
2016-01-14 17:16:37,948 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:37,949 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791797897_0001_000000, startTime=1452791797949
2016-01-14 17:16:37,949 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:37,949 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:38,023 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:38,023 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:38,023 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...lyplatform/0_UNKNOWN_SWAP, applicationId=application_1452791797897_0001, dagName=[E4B14FA96F2E491FBD6D0B46F61504BF/F527F4C49E52423FA3C5013B32FC4EF5] every/(1/1) ...lyplatform/0_UNKNOWN_SWAP
2016-01-14 17:16:38,038 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [E4B14FA96F2E491FBD6D0B46F61504BF/F527F4C49E52423FA3C5013B32FC4EF5] every/(1/1) ...lyplatform/0_UNKNOWN_SWAP
2016-01-14 17:16:38,039 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791797897_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E4B14FA96F2E491FBD6D0B46F61504BF/F527F4C49E52423FA3C5013B32FC4EF5/.tez/application_1452791797897_0001/localmode-log-dir/dag_1452791797897_0001_1.dot
2016-01-14 17:16:38,039 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E4B14FA96F2E491FBD6D0B46F61504BF/F527F4C49E52423FA3C5013B32FC4EF5/.tez/application_1452791797897_0001/localmode-log-dir/dag_1452791797897_0001_1-tez-dag.pb.txt
2016-01-14 17:16:38,045 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791797897_0001_1
2016-01-14 17:16:38,045 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:38,045 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [E4B14FA96F2E491FBD6D0B46F61504BF/F527F4C49E52423FA3C5013B32FC4EF5] every/(1/1) ...lyplatform/0_UNKNOWN_SWAP
2016-01-14 17:16:38 Running Dag: dag_1452791797897_0001_1
2016-01-14 17:16:38,057 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:38,058 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791797897_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791797897_0001_1, submitTime=1452791798038
2016-01-14 17:16:38,059 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex ED22ED96BD5B4AF494E8DB0887591CCA
2016-01-14 17:16:38,059 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:38,059 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:38,059 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791797897_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791797897_0001_1, initTime=1452791798058
2016-01-14 17:16:38,060 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791797897_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:38,060 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791797897_0001_1][Event:DAG_STARTED]: dagID=dag_1452791797897_0001_1, startTime=1452791798060
2016-01-14 17:16:38,060 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:38,060 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791797897_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:38,060 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: F492D6A6A3964A94BEDB73B7793F87DF : {08C68483485943B69ABF00F63033D3E5={InputName=08C68483485943B69ABF00F63033D3E5}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:38,060 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 08C68483485943B69ABF00F63033D3E5, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:38,060 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791797897_0001_1_00 [F492D6A6A3964A94BEDB73B7793F87DF]
2016-01-14 17:16:38,060 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791797897_0001_1_00 [F492D6A6A3964A94BEDB73B7793F87DF]
2016-01-14 17:16:38,060 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:38,060 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791797897_0001_1_00 [F492D6A6A3964A94BEDB73B7793F87DF]
2016-01-14 17:16:38,061 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791797897_0001_1_00 [F492D6A6A3964A94BEDB73B7793F87DF]. Starting root input initializers: 1
2016-01-14 17:16:38,061 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791797897_0001_1_00 [F492D6A6A3964A94BEDB73B7793F87DF] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:38,061 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791797897_0001_1_01 [ED22ED96BD5B4AF494E8DB0887591CCA]
2016-01-14 17:16:38,061 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...lyplatform/0_UNKNOWN_SWAP, applicationId=application_1452791797897_0001, dagName=[E4B14FA96F2E491FBD6D0B46F61504BF/F527F4C49E52423FA3C5013B32FC4EF5] every/(1/1) ...lyplatform/0_UNKNOWN_SWAP
2016-01-14 17:16:38,062 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 08C68483485943B69ABF00F63033D3E5 on vertex vertex_1452791797897_0001_1_00 [F492D6A6A3964A94BEDB73B7793F87DF]
2016-01-14 17:16:38,063 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:38,063 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 08C68483485943B69ABF00F63033D3E5 asking for 1 tasks. Headroom: 714 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:38,064 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:38,064 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:38,064 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:38,065 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791797897_0001_1_01 [ED22ED96BD5B4AF494E8DB0887591CCA]
2016-01-14 17:16:38,066 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791797897_0001_1_01 [ED22ED96BD5B4AF494E8DB0887591CCA]
2016-01-14 17:16:38,066 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791797897_0001_1_01 [ED22ED96BD5B4AF494E8DB0887591CCA]
2016-01-14 17:16:38,066 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=7B344E8334D84E5BB99E12B65D8C1158, vertexId=vertex_1452791797897_0001_1_01 [ED22ED96BD5B4AF494E8DB0887591CCA], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:38,066 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:38,066 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=7B344E8334D84E5BB99E12B65D8C1158, vertexId=vertex_1452791797897_0001_1_01 [ED22ED96BD5B4AF494E8DB0887591CCA]
2016-01-14 17:16:38,066 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 08C68483485943B69ABF00F63033D3E5 on vertex vertex_1452791797897_0001_1_00 [F492D6A6A3964A94BEDB73B7793F87DF]
2016-01-14 17:16:38,068 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=7B344E8334D84E5BB99E12B65D8C1158, vertexName=ED22ED96BD5B4AF494E8DB0887591CCA, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:38,068 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=7B344E8334D84E5BB99E12B65D8C1158, vertexId=vertex_1452791797897_0001_1_01 [ED22ED96BD5B4AF494E8DB0887591CCA]
2016-01-14 17:16:38,069 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:38,069 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791797897_0001_1][Event:VERTEX_INITIALIZED]: vertexName=ED22ED96BD5B4AF494E8DB0887591CCA, vertexId=vertex_1452791797897_0001_1_01, initRequestedTime=1452791798061, initedTime=1452791798069, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:38,070 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791797897_0001_1_01 [ED22ED96BD5B4AF494E8DB0887591CCA] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:38,071 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791797897_0001_1_00 [F492D6A6A3964A94BEDB73B7793F87DF] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:38,071 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {08C68483485943B69ABF00F63033D3E5=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:38,071 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791797897_0001_1_00 [F492D6A6A3964A94BEDB73B7793F87DF] parallelism set to 1
2016-01-14 17:16:38,071 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791797897_0001_1_00 [F492D6A6A3964A94BEDB73B7793F87DF] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:38,071 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791797897_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791797897_0001_1_00, eventCount=1
2016-01-14 17:16:38,072 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:38,072 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791797897_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F492D6A6A3964A94BEDB73B7793F87DF, vertexId=vertex_1452791797897_0001_1_00, initRequestedTime=1452791798060, initedTime=1452791798072, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:38,072 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791797897_0001_1_00 [F492D6A6A3964A94BEDB73B7793F87DF] with distanceFromRoot: 0
2016-01-14 17:16:38,072 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791797897_0001_1_00 [F492D6A6A3964A94BEDB73B7793F87DF] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:38,072 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in F492D6A6A3964A94BEDB73B7793F87DF
2016-01-14 17:16:38,073 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791797897_0001_1_00 [F492D6A6A3964A94BEDB73B7793F87DF]
2016-01-14 17:16:38,073 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:38,073 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791797897_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791797897_0001_1_00, startRequestedTime=1452791798072, startedTime=1452791798072
2016-01-14 17:16:38,073 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791797897_0001_1_00 [F492D6A6A3964A94BEDB73B7793F87DF] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:38,074 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:38,074 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791797897_0001_1][Event:TASK_STARTED]: vertexName=F492D6A6A3964A94BEDB73B7793F87DF, taskId=task_1452791797897_0001_1_00_000000, scheduledTime=1452791798074, launchTime=1452791798074
2016-01-14 17:16:38,074 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791797897_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:38,074 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791797897_0001_1_00 for vertex: vertex_1452791797897_0001_1_01 [ED22ED96BD5B4AF494E8DB0887591CCA] numStartedSources: 1 numSources: 1
2016-01-14 17:16:38,075 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791797897_0001_1_01 [ED22ED96BD5B4AF494E8DB0887591CCA] with distanceFromRoot: 1
2016-01-14 17:16:38,075 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791797897_0001_1_00 [F492D6A6A3964A94BEDB73B7793F87DF]
2016-01-14 17:16:38,075 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: F492D6A6A3964A94BEDB73B7793F87DF in vertex: ED22ED96BD5B4AF494E8DB0887591CCA
2016-01-14 17:16:38,076 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: ED22ED96BD5B4AF494E8DB0887591CCA with 1 source tasks and 1 pending tasks
2016-01-14 17:16:38,076 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:38,076 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791797897_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791797897_0001_1_01, startRequestedTime=1452791798075, startedTime=1452791798075
2016-01-14 17:16:38,076 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791797897_0001_1_01 [ED22ED96BD5B4AF494E8DB0887591CCA] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:38,077 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [E4B14FA96F2E491FBD6D0B46F61504BF/F527F4C49E52423FA3C5013B32FC4EF5] every/(1/1) ...lyplatform/0_UNKNOWN_SWAP, VertexName: F492D6A6A3964A94BEDB73B7793F87DF, VertexParallelism: 1, TaskAttemptID:attempt_1452791797897_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=08C68483485943B69ABF00F63033D3E5, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=ED22ED96BD5B4AF494E8DB0887591CCA, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:38,077 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791797897_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:38,077 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:38,078 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:38,078 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:38,079 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:38,079 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791797897_0001_00_000001
2016-01-14 17:16:38,079 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791797897_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:38,080 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791797897_0001_00_000001
2016-01-14 17:16:38,080 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:38,081 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791797897_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:38,081 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:38,081 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791797897_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791797897_0001_00_000001, launchTime=1452791798081
2016-01-14 17:16:38,084 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791797897_0001_00_000001
2016-01-14 17:16:38,084 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791797897_0001_1_00_000000_0] to container: [container_1452791797897_0001_00_000001]
2016-01-14 17:16:38,084 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791797897_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:38,085 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791797897_0001_00_000001 given task: attempt_1452791797897_0001_1_00_000000_0
2016-01-14 17:16:38,085 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791797897_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791797897_0001_1_00_000000_0
2016-01-14 17:16:38,085 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791797897_0001_1_00_000000_0] started. Is using containerId: [container_1452791797897_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:38,085 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:38,085 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791797897_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F492D6A6A3964A94BEDB73B7793F87DF, taskAttemptId=attempt_1452791797897_0001_1_00_000000_0, startTime=1452791798085, containerId=container_1452791797897_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791797897_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:38,086 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791797897_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:38,086 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791797897_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:38,086 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791797897_0001, with dag id: dag_1452791797897_0001_1
2016-01-14 17:16:38 Starting to run new task attempt: attempt_1452791797897_0001_1_00_000000_0
2016-01-14 17:16:38,087 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791797897_0001_1_00_000000_0
2016-01-14 17:16:38,087 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:38,087 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:38,088 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:38,088 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:38,088 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:38,088 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:38,089 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [E4B14FA96F2E491FBD6D0B46F61504BF/F527F4C49E52423FA3C5013B32FC4EF5] every/(1/1) ...lyplatform/0_UNKNOWN_SWAP, VertexName: F492D6A6A3964A94BEDB73B7793F87DF, VertexParallelism: 1, TaskAttemptID:attempt_1452791797897_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=08C68483485943B69ABF00F63033D3E5, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=ED22ED96BD5B4AF494E8DB0887591CCA, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:38,089 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=749207552
2016-01-14 17:16:38,089 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791797897_0001_1_00_000000_0
2016-01-14 17:16:38,090 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:38,090 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:38,090 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=08C68483485943B69ABF00F63033D3E5, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:38,090 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791797897_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:38,091 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:38,091 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=ED22ED96BD5B4AF494E8DB0887591CCA, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:38,091 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:38,091 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 08C68483485943B69ABF00F63033D3E5
2016-01-14 17:16:38,092 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 08C68483485943B69ABF00F63033D3E5
2016-01-14 17:16:38,093 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:38,094 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:38,094 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:38,094 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 08C68483485943B69ABF00F63033D3E5
2016-01-14 17:16:38,094 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 08C68483485943B69ABF00F63033D3E5
2016-01-14 17:16:38,094 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: ED22ED96BD5B4AF494E8DB0887591CCA
2016-01-14 17:16:38,097 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:38,097 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: ED22ED96BD5B4AF494E8DB0887591CCA
2016-01-14 17:16:38,097 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F492D6A6A3964A94BEDB73B7793F87DF, ordinal: 0
2016-01-14 17:16:38,098 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F492D6A6A3964A94BEDB73B7793F87DF, mem on start (mb), free: 288, total: 714, max: 714
2016-01-14 17:16:38,098 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:38,097 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: ED22ED96BD5B4AF494E8DB0887591CCA
2016-01-14 17:16:38,098 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:38,098 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:38,098 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:38,098 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:38,099 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:38,099 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:38,099 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:38,099 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:38,099 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 749207552, TotalAvailable: 501969059, TotalRequested/TotalJVMHeap:0.14
2016-01-14 17:16:38,099 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 08C68483485943B69ABF00F63033D3E5, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:38,100 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, ED22ED96BD5B4AF494E8DB0887591CCA, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:38,100 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:38,100 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 08C68483485943B69ABF00F63033D3E5 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:38,100 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:38,100 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:38,100 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 08C68483485943B69ABF00F63033D3E5
2016-01-14 17:16:38,100 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:38,101 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 08C68483485943B69ABF00F63033D3E5
2016-01-14 17:16:38,101 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:38,101 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791797897_0001_1_00_000000_0
2016-01-14 17:16:38,102 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:38,102 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@51148f22
2016-01-14 17:16:38,106 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:38,106 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@51148f22
2016-01-14 17:16:38,106 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:38,107 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:38,106 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 08C68483485943B69ABF00F63033D3E5
2016-01-14 17:16:38,107 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 484C58C53458462881C3FB9181FFC84B
2016-01-14 17:16:38,108 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 484C58C53458462881C3FB9181FFC84B
2016-01-14 17:16:38,108 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@fc975d; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@47355b2c; comparator=cascading.tuple.hadoop.util.TupleComparator@7ff4ca39; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:38,108 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:38,108 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:38,109 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:38,136 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:38,136 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:38,136 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:38,137 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:38,137 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:38,137 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:38,137 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:38,138 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F492D6A6A3964A94BEDB73B7793F87DF, all 1 inputs ready in: 00:00:00.001
2016-01-14 17:16:38,141 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791797897_0001_1_00_000000_0
2016-01-14 17:16:38,142 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F492D6A6A3964A94BEDB73B7793F87DF, mem on close (mb), free: 373, total: 717, max: 717
2016-01-14 17:16:38,142 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:38,142 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:38,142 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:38,142 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:38,146 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:38,147 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 17:16:38,147 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 17:16:38 Completed running task attempt: attempt_1452791797897_0001_1_00_000000_0
2016-01-14 17:16:38,148 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791797897_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:38,149 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791797897_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791797897_0001_1_00, eventCount=1
2016-01-14 17:16:38,149 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791797897_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F492D6A6A3964A94BEDB73B7793F87DF, taskAttemptId=attempt_1452791797897_0001_1_00_000000_0, startTime=1452791798085, finishTime=1452791798149, timeTaken=64, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=15, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751828992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791798138, Process_Duration=3, Process_End_Time=1452791798141, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:38,149 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:38,149 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791797897_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:38,149 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791797897_0001_1_00_000000_0
2016-01-14 17:16:38,150 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:38,150 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:38,150 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791797897_0001_1][Event:TASK_FINISHED]: vertexName=F492D6A6A3964A94BEDB73B7793F87DF, taskId=task_1452791797897_0001_1_00_000000, startTime=1452791798085, finishTime=1452791798149, timeTaken=64, status=SUCCEEDED, successfulAttemptID=attempt_1452791797897_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=15, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751828992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791798138, Process_Duration=3, Process_End_Time=1452791798141, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:38,150 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791797897_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:38,151 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791797897_0001_1_00 [F492D6A6A3964A94BEDB73B7793F87DF] : 1
2016-01-14 17:16:38,151 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791797897_0001_1_00 [F492D6A6A3964A94BEDB73B7793F87DF]
2016-01-14 17:16:38,151 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:38,152 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791797897_0001_1][Event:VERTEX_FINISHED]: vertexName=F492D6A6A3964A94BEDB73B7793F87DF, vertexId=vertex_1452791797897_0001_1_00, initRequestedTime=1452791798060, initedTime=1452791798072, startRequestedTime=1452791798072, startedTime=1452791798072, finishTime=1452791798151, timeTaken=79, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=15, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751828992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791798138, Process_Duration=3, Process_End_Time=1452791798141, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791798085, firstTasksToStart=[ task_1452791797897_0001_1_00_000000 ], lastTaskFinishTime=1452791798149, lastTasksToFinish=[ task_1452791797897_0001_1_00_000000 ], minTaskDuration=64, maxTaskDuration=64, avgTaskDuration=64.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791797897_0001_1_00_000000 ], longestDurationTasks=[ task_1452791797897_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:38,152 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791797897_0001_1_00 [F492D6A6A3964A94BEDB73B7793F87DF] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:38,152 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791797897_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:38,152 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791797897_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791797897_0001_00_000001, stoppedTime=1452791798152, exitStatus=0
2016-01-14 17:16:38,152 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791797897_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:38,153 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791797897_0001_1_01 [ED22ED96BD5B4AF494E8DB0887591CCA] attempt: attempt_1452791797897_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:38,153 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: ED22ED96BD5B4AF494E8DB0887591CCA
2016-01-14 17:16:38,153 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791797897_0001_1_01 [ED22ED96BD5B4AF494E8DB0887591CCA]
2016-01-14 17:16:38,153 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791797897_0001_1_00 [F492D6A6A3964A94BEDB73B7793F87DF] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:38,152 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:38,153 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791797897_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:38,154 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791797897_0001_1][Event:TASK_STARTED]: vertexName=ED22ED96BD5B4AF494E8DB0887591CCA, taskId=task_1452791797897_0001_1_01_000000, scheduledTime=1452791798154, launchTime=1452791798154
2016-01-14 17:16:38,154 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791797897_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:38,154 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791797897_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:38,154 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791797897_0001_1_01 [ED22ED96BD5B4AF494E8DB0887591CCA]
2016-01-14 17:16:38,154 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [E4B14FA96F2E491FBD6D0B46F61504BF/F527F4C49E52423FA3C5013B32FC4EF5] every/(1/1) ...lyplatform/0_UNKNOWN_SWAP, VertexName: ED22ED96BD5B4AF494E8DB0887591CCA, VertexParallelism: 1, TaskAttemptID:attempt_1452791797897_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F492D6A6A3964A94BEDB73B7793F87DF, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=7B344E8334D84E5BB99E12B65D8C1158, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:38,154 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:38,154 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791797897_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:38,154 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:38,154 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=15, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751828992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791798138, Process_Duration=3, Process_End_Time=1452791798141, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:38,155 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791797897_0001_00_000002
2016-01-14 17:16:38,156 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791797897_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:38,156 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:38,157 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:38,157 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791797897_0001_00_000001
2016-01-14 17:16:38,157 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791797897_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:38,157 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791797897_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:38,157 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791797897_0001_00_000001, Exiting
2016-01-14 17:16:38,157 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791797897_0001_00_000001
2016-01-14 17:16:38,157 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791797897_0001_00_000001
2016-01-14 17:16:38,158 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791797897_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:38,159 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791797897_0001_00_000001 completed successfully
2016-01-14 17:16:38,159 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791797897_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:38,159 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791797897_0001_00_000002
2016-01-14 17:16:38,160 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 17:16:38,161 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791797897_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:38,162 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791797897_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791797897_0001_00_000002, launchTime=1452791798161
2016-01-14 17:16:38,161 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:38,162 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791797897_0001_00_000002
2016-01-14 17:16:38,162 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791797897_0001_1_01_000000_0] to container: [container_1452791797897_0001_00_000002]
2016-01-14 17:16:38,162 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791797897_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:38,163 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791797897_0001_1_01_000000_0] started. Is using containerId: [container_1452791797897_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:38,163 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:38,163 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791797897_0001_00_000002 given task: attempt_1452791797897_0001_1_01_000000_0
2016-01-14 17:16:38,163 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791797897_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=ED22ED96BD5B4AF494E8DB0887591CCA, taskAttemptId=attempt_1452791797897_0001_1_01_000000_0, startTime=1452791798163, containerId=container_1452791797897_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791797897_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:38,163 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791797897_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:38,163 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791797897_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:38,163 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791797897_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791797897_0001_1_01_000000_0
2016-01-14 17:16:38 Starting to run new task attempt: attempt_1452791797897_0001_1_01_000000_0
2016-01-14 17:16:38,164 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791797897_0001_1_01_000000_0
2016-01-14 17:16:38,164 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:38,164 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:38,164 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:38,164 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:38,164 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:38,164 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:38,164 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [E4B14FA96F2E491FBD6D0B46F61504BF/F527F4C49E52423FA3C5013B32FC4EF5] every/(1/1) ...lyplatform/0_UNKNOWN_SWAP, VertexName: ED22ED96BD5B4AF494E8DB0887591CCA, VertexParallelism: 1, TaskAttemptID:attempt_1452791797897_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F492D6A6A3964A94BEDB73B7793F87DF, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=7B344E8334D84E5BB99E12B65D8C1158, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:38,164 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=751828992
2016-01-14 17:16:38,165 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791797897_0001_1_01_000000_0
2016-01-14 17:16:38,165 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:38,165 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:38,165 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F492D6A6A3964A94BEDB73B7793F87DF, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:38,165 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791797897_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:38,166 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:38,166 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F492D6A6A3964A94BEDB73B7793F87DF
2016-01-14 17:16:38,166 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=7B344E8334D84E5BB99E12B65D8C1158, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:38,166 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:38,168 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 676646080, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:38,168 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:38,168 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 676646080, type: INPUT, componentVertexName: F492D6A6A3964A94BEDB73B7793F87DF
2016-01-14 17:16:38,168 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F492D6A6A3964A94BEDB73B7793F87DF
2016-01-14 17:16:38,169 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:38,170 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:38,170 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 7B344E8334D84E5BB99E12B65D8C1158
2016-01-14 17:16:38,170 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:38,170 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 7B344E8334D84E5BB99E12B65D8C1158
2016-01-14 17:16:38,176 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: ED22ED96BD5B4AF494E8DB0887591CCA, ordinal: 1
2016-01-14 17:16:38,176 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: ED22ED96BD5B4AF494E8DB0887591CCA, mem on start (mb), free: 367, total: 717, max: 717
2016-01-14 17:16:38,176 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:38,176 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:38,176 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:38,180 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:38,180 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 7B344E8334D84E5BB99E12B65D8C1158
2016-01-14 17:16:38,180 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:38,180 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:38,180 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:38,181 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:38,181 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:38,181 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:38,181 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 676646080, TotalRequestedScaled: 6.245963815384616E8, TotalJVMHeap: 751828992, TotalAvailable: 503725424, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:38,182 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F492D6A6A3964A94BEDB73B7793F87DF, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=676646080, allocated=503725424
2016-01-14 17:16:38,182 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 7B344E8334D84E5BB99E12B65D8C1158, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:38,182 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:38,182 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F492D6A6A3964A94BEDB73B7793F87DF being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:38,182 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F492D6A6A3964A94BEDB73B7793F87DF
2016-01-14 17:16:38,183 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:38,183 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:38,183 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: F492D6A6A3964A94BEDB73B7793F87DF with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:38,183 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=676646080, postMergeMem=0, RuntimeTotalAvailable=503725424. Updated to: ShuffleMem=503725424, postMergeMem: 0
2016-01-14 17:16:38,184 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=503725424, maxSingleShuffleLimit=125931360, mergeThreshold=453352896, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:38,184 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:38,184 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:38,184 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:38,184 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F492D6A6A3964A94BEDB73B7793F87DF
2016-01-14 17:16:38,185 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:38,186 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791797897_0001_1_01_000000_0
2016-01-14 17:16:38,187 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791797897_0001_1_00_000000_0_10171, runDuration: 53016, ]
2016-01-14 17:16:38,188 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791797897_0001_1_00_000000_0_10171] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791798188, TimeTaken=0, Rate=0.00 MB/s
2016-01-14 17:16:38,189 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:38,189 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : F492D6A6A3964A94BEDB73B7793F87DF
2016-01-14 17:16:38,189 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:38,189 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [F492D6A6A3964A94BEDB73B7793F87DF] #29 in 1ms
2016-01-14 17:16:38,189 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 484C58C53458462881C3FB9181FFC84B
2016-01-14 17:16:38,189 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [F492D6A6A3964A94BEDB73B7793F87DF] #29, status:false, isInterrupted:false
2016-01-14 17:16:38,190 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:38,190 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:38,190 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:38,190 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:38,190 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:38,191 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : F492D6A6A3964A94BEDB73B7793F87DF
2016-01-14 17:16:38,191 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:38,189 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_SWAP"], id: 7B344E8334D84E5BB99E12B65D8C1158
2016-01-14 17:16:38,191 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_SWAP"]
2016-01-14 17:16:38,191 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 484C58C53458462881C3FB9181FFC84B, for 1 inputs
2016-01-14 17:16:38,191 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: ED22ED96BD5B4AF494E8DB0887591CCA, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:38,192 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@431d2459; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@2e3d717b
2016-01-14 17:16:38,196 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:38,197 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:38,197 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791797897_0001_1_01_000000_0
2016-01-14 17:16:38,197 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791797897_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:38,197 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791797897_0001_1_01_000000_0
2016-01-14 17:16:38,197 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: ED22ED96BD5B4AF494E8DB0887591CCA, mem on close (mb), free: 365, total: 717, max: 717
2016-01-14 17:16:38,197 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: F492D6A6A3964A94BEDB73B7793F87DF
2016-01-14 17:16:38,199 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:38,200 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:38,200 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:38,200 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:38,200 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:38 Completed running task attempt: attempt_1452791797897_0001_1_01_000000_0
2016-01-14 17:16:38,200 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791797897_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:38,201 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=183, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751828992, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791798191, Process_Duration=6, Process_End_Time=1452791798197, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 17:16:38,201 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:38,201 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:38,201 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791797897_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=ED22ED96BD5B4AF494E8DB0887591CCA, taskAttemptId=attempt_1452791797897_0001_1_01_000000_0, startTime=1452791798163, finishTime=1452791798201, timeTaken=38, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=183, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751828992, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791798191, Process_Duration=6, Process_End_Time=1452791798197, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:38,201 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:38,201 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791797897_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:38,201 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791797897_0001_00_000002
2016-01-14 17:16:38,201 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791797897_0001_1_01_000000_0
2016-01-14 17:16:38,201 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:38,202 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:38,202 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791797897_0001_1][Event:TASK_FINISHED]: vertexName=ED22ED96BD5B4AF494E8DB0887591CCA, taskId=task_1452791797897_0001_1_01_000000, startTime=1452791798163, finishTime=1452791798202, timeTaken=39, status=SUCCEEDED, successfulAttemptID=attempt_1452791797897_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=183, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751828992, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791798191, Process_Duration=6, Process_End_Time=1452791798197, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:38,202 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791797897_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:38,202 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791797897_0001_1_01 [ED22ED96BD5B4AF494E8DB0887591CCA] : 1
2016-01-14 17:16:38,202 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791797897_0001_1_01 [ED22ED96BD5B4AF494E8DB0887591CCA]
2016-01-14 17:16:38,203 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:38,203 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791797897_0001_1][Event:VERTEX_FINISHED]: vertexName=ED22ED96BD5B4AF494E8DB0887591CCA, vertexId=vertex_1452791797897_0001_1_01, initRequestedTime=1452791798061, initedTime=1452791798069, startRequestedTime=1452791798075, startedTime=1452791798075, finishTime=1452791798202, timeTaken=127, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=183, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751828992, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791798191, Process_Duration=6, Process_End_Time=1452791798197, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452791798163, firstTasksToStart=[ task_1452791797897_0001_1_01_000000 ], lastTaskFinishTime=1452791798201, lastTasksToFinish=[ task_1452791797897_0001_1_01_000000 ], minTaskDuration=38, maxTaskDuration=38, avgTaskDuration=38.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791797897_0001_1_01_000000 ], longestDurationTasks=[ task_1452791797897_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:38,203 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791797897_0001_1_01 [ED22ED96BD5B4AF494E8DB0887591CCA] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:38,203 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791797897_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:38,204 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791797897_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791797897_0001_00_000002, stoppedTime=1452791798203, exitStatus=0
2016-01-14 17:16:38,204 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:38,204 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791797897_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:38,204 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791797897_0001_1_01 [ED22ED96BD5B4AF494E8DB0887591CCA] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:38,204 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791797897_0001_1
2016-01-14 17:16:38,204 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791797897_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452791797897_0001_1
2016-01-14 17:16:38,204 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 7B344E8334D84E5BB99E12B65D8C1158 for vertex: vertex_1452791797897_0001_1_01 [ED22ED96BD5B4AF494E8DB0887591CCA]
2016-01-14 17:16:38,210 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452791797897_0001_1_00 [F492D6A6A3964A94BEDB73B7793F87DF]
2016-01-14 17:16:38,212 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791797897_0001_1, queueSize=0
2016-01-14 17:16:38,212 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791797897_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 17:16:38,212 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:38,212 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791797897_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:38,212 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791797897_0001_00_000002, Exiting
2016-01-14 17:16:38,212 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791797897_0001_00_000002
2016-01-14 17:16:38,212 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791797897_0001_00_000002
2016-01-14 17:16:38,213 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791797897_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:38,213 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791797897_0001_00_000002 completed successfully
2016-01-14 17:16:38,213 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791797897_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791797897_0001_1, startTime=1452791798060, finishTime=1452791798211, timeTaken=151, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=516, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=15, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1503657984, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905583596329, Process_Duration=9, Process_End_Time=2905583596338, Read_Duration=4, Tuples_Read=40, Tuples_Written=40, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 17:16:38,213 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791797897_0001_1 finished with state: SUCCEEDED
2016-01-14 17:16:38,213 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791797897_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:38,213 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452791797897_0001_00_000002
2016-01-14 17:16:38,213 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791797897_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:16:38 Completed Dag: dag_1452791797897_0001_1
2016-01-14 17:16:38,214 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791797897_0001_1, dagState=SUCCEEDED
2016-01-14 17:16:38,214 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791797897_0001_1_post
2016-01-14 17:16:38,214 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:38,214 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:38,220 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...lyplatform/0_UNKNOWN_SWAP, applicationId=application_1452791797897_0001
2016-01-14 17:16:38,221 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:38,221 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:38,221 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:38,221 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:38,221 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:38,222 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:38,222 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:38,222 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:38,222 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:38,222 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:38,222 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:38,222 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:38,223 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:38,226 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E4B14FA96F2E491FBD6D0B46F61504BF/F527F4C49E52423FA3C5013B32FC4EF5/.tez/application_1452791797897_0001
2016-01-14 17:16:38,227 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.347
2016-01-14 17:16:38,290 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_UNKNOWN_SWAP
2016-01-14 17:16:38,311 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:38,311 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:38,346 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.011
2016-01-14 17:16:38,352 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.016
2016-01-14 17:16:38,352 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:38,352 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:38,352 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:38,364 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:38,369 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_-1
2016-01-14 17:16:38,411 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:38,411 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:38,411 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_-1"]
2016-01-14 17:16:38,411 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:38,411 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:38,411 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:38,412 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...emblyplatform/0_VALUES_-1
2016-01-14 17:16:38,412 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:38,422 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:38,427 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:38,427 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:38,427 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:38,436 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/0FFF661DC8274DA79B10D3BAFD3990C2/B290CE9CF7D246E2B14BC587B5CE2C8B/.tez/application_1452791798427_0001 doesn't exist and is created
2016-01-14 17:16:38,453 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:38,453 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:38,454 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/0FFF661DC8274DA79B10D3BAFD3990C2/B290CE9CF7D246E2B14BC587B5CE2C8B/.tez/application_1452791798427_0001
2016-01-14 17:16:38,457 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791798427_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:38,466 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:38,467 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:38,468 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:38,468 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:38,469 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/0FFF661DC8274DA79B10D3BAFD3990C2/B290CE9CF7D246E2B14BC587B5CE2C8B/.tez/application_1452791798427_0001/localmode-log-dir
2016-01-14 17:16:38,469 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/0FFF661DC8274DA79B10D3BAFD3990C2/B290CE9CF7D246E2B14BC587B5CE2C8B/.tez/application_1452791798427_0001/localmode-log-dir/history.txt.appattempt_1452791798427_0001_000000, maxErrors=10
2016-01-14 17:16:38,469 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:38,469 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791798427_0001_000000, appSubmitTime=1452791798456, launchTime=1452791798456
2016-01-14 17:16:38,474 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:38,477 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:48999
2016-01-14 17:16:38,478 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:38,479 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:38,479 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:38,480 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:38,481 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791798427_0001_000000, startTime=1452791798481
2016-01-14 17:16:38,481 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:38,482 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:38,553 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:38,553 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:38,554 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...emblyplatform/0_VALUES_-1, applicationId=application_1452791798427_0001, dagName=[0FFF661DC8274DA79B10D3BAFD3990C2/B290CE9CF7D246E2B14BC587B5CE2C8B] every/(1/1) ...emblyplatform/0_VALUES_-1
2016-01-14 17:16:38,569 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [0FFF661DC8274DA79B10D3BAFD3990C2/B290CE9CF7D246E2B14BC587B5CE2C8B] every/(1/1) ...emblyplatform/0_VALUES_-1
2016-01-14 17:16:38,570 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791798427_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/0FFF661DC8274DA79B10D3BAFD3990C2/B290CE9CF7D246E2B14BC587B5CE2C8B/.tez/application_1452791798427_0001/localmode-log-dir/dag_1452791798427_0001_1.dot
2016-01-14 17:16:38,571 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/0FFF661DC8274DA79B10D3BAFD3990C2/B290CE9CF7D246E2B14BC587B5CE2C8B/.tez/application_1452791798427_0001/localmode-log-dir/dag_1452791798427_0001_1-tez-dag.pb.txt
2016-01-14 17:16:38,576 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791798427_0001_1
2016-01-14 17:16:38,576 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:38,576 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [0FFF661DC8274DA79B10D3BAFD3990C2/B290CE9CF7D246E2B14BC587B5CE2C8B] every/(1/1) ...emblyplatform/0_VALUES_-1
2016-01-14 17:16:38 Running Dag: dag_1452791798427_0001_1
2016-01-14 17:16:38,588 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:38,588 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791798427_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791798427_0001_1, submitTime=1452791798569
2016-01-14 17:16:38,589 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex E6E183917E994AF98262AB6649D7AF69
2016-01-14 17:16:38,589 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:38,589 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:38,589 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791798427_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791798427_0001_1, initTime=1452791798588
2016-01-14 17:16:38,589 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791798427_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:38,589 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791798427_0001_1][Event:DAG_STARTED]: dagID=dag_1452791798427_0001_1, startTime=1452791798589
2016-01-14 17:16:38,590 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:38,590 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791798427_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:38,590 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: A620AFF2E7154A089412E06930D66D90 : {EDE301EC31844B4A97E06348C7C8E8AE={InputName=EDE301EC31844B4A97E06348C7C8E8AE}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:38,590 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: EDE301EC31844B4A97E06348C7C8E8AE, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:38,590 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791798427_0001_1_00 [A620AFF2E7154A089412E06930D66D90]
2016-01-14 17:16:38,590 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:38,590 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791798427_0001_1_00 [A620AFF2E7154A089412E06930D66D90]
2016-01-14 17:16:38,590 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791798427_0001_1_00 [A620AFF2E7154A089412E06930D66D90]
2016-01-14 17:16:38,590 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791798427_0001_1_00 [A620AFF2E7154A089412E06930D66D90]. Starting root input initializers: 1
2016-01-14 17:16:38,591 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791798427_0001_1_00 [A620AFF2E7154A089412E06930D66D90] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:38,591 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...emblyplatform/0_VALUES_-1, applicationId=application_1452791798427_0001, dagName=[0FFF661DC8274DA79B10D3BAFD3990C2/B290CE9CF7D246E2B14BC587B5CE2C8B] every/(1/1) ...emblyplatform/0_VALUES_-1
2016-01-14 17:16:38,591 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791798427_0001_1_01 [E6E183917E994AF98262AB6649D7AF69]
2016-01-14 17:16:38,591 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: EDE301EC31844B4A97E06348C7C8E8AE on vertex vertex_1452791798427_0001_1_00 [A620AFF2E7154A089412E06930D66D90]
2016-01-14 17:16:38,593 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:38,593 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input EDE301EC31844B4A97E06348C7C8E8AE asking for 1 tasks. Headroom: 717 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:38,594 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:38,594 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:38,594 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:38,594 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791798427_0001_1_01 [E6E183917E994AF98262AB6649D7AF69]
2016-01-14 17:16:38,594 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791798427_0001_1_01 [E6E183917E994AF98262AB6649D7AF69]
2016-01-14 17:16:38,594 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791798427_0001_1_01 [E6E183917E994AF98262AB6649D7AF69]
2016-01-14 17:16:38,594 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=36C44688EBA544D7A931ED9BDF10799C, vertexId=vertex_1452791798427_0001_1_01 [E6E183917E994AF98262AB6649D7AF69], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:38,595 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=36C44688EBA544D7A931ED9BDF10799C, vertexId=vertex_1452791798427_0001_1_01 [E6E183917E994AF98262AB6649D7AF69]
2016-01-14 17:16:38,596 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:38,596 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: EDE301EC31844B4A97E06348C7C8E8AE on vertex vertex_1452791798427_0001_1_00 [A620AFF2E7154A089412E06930D66D90]
2016-01-14 17:16:38,596 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=36C44688EBA544D7A931ED9BDF10799C, vertexName=E6E183917E994AF98262AB6649D7AF69, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:38,596 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=36C44688EBA544D7A931ED9BDF10799C, vertexId=vertex_1452791798427_0001_1_01 [E6E183917E994AF98262AB6649D7AF69]
2016-01-14 17:16:38,597 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791798427_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E6E183917E994AF98262AB6649D7AF69, vertexId=vertex_1452791798427_0001_1_01, initRequestedTime=1452791798591, initedTime=1452791798597, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:38,597 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:38,597 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791798427_0001_1_01 [E6E183917E994AF98262AB6649D7AF69] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:38,598 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791798427_0001_1_00 [A620AFF2E7154A089412E06930D66D90] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:38,598 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {EDE301EC31844B4A97E06348C7C8E8AE=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:38,598 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791798427_0001_1_00 [A620AFF2E7154A089412E06930D66D90] parallelism set to 1
2016-01-14 17:16:38,598 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791798427_0001_1_00 [A620AFF2E7154A089412E06930D66D90] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:38,598 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791798427_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791798427_0001_1_00, eventCount=1
2016-01-14 17:16:38,599 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791798427_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A620AFF2E7154A089412E06930D66D90, vertexId=vertex_1452791798427_0001_1_00, initRequestedTime=1452791798590, initedTime=1452791798599, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:38,599 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:38,599 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791798427_0001_1_00 [A620AFF2E7154A089412E06930D66D90] with distanceFromRoot: 0
2016-01-14 17:16:38,599 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791798427_0001_1_00 [A620AFF2E7154A089412E06930D66D90] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:38,599 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in A620AFF2E7154A089412E06930D66D90
2016-01-14 17:16:38,600 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791798427_0001_1_00 [A620AFF2E7154A089412E06930D66D90]
2016-01-14 17:16:38,600 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:38,600 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791798427_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791798427_0001_1_00, startRequestedTime=1452791798599, startedTime=1452791798599
2016-01-14 17:16:38,600 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791798427_0001_1_00 [A620AFF2E7154A089412E06930D66D90] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:38,601 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:38,601 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791798427_0001_1][Event:TASK_STARTED]: vertexName=A620AFF2E7154A089412E06930D66D90, taskId=task_1452791798427_0001_1_00_000000, scheduledTime=1452791798601, launchTime=1452791798601
2016-01-14 17:16:38,601 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791798427_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:38,601 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791798427_0001_1_00 for vertex: vertex_1452791798427_0001_1_01 [E6E183917E994AF98262AB6649D7AF69] numStartedSources: 1 numSources: 1
2016-01-14 17:16:38,601 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791798427_0001_1_01 [E6E183917E994AF98262AB6649D7AF69] with distanceFromRoot: 1
2016-01-14 17:16:38,602 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791798427_0001_1_00 [A620AFF2E7154A089412E06930D66D90]
2016-01-14 17:16:38,602 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: A620AFF2E7154A089412E06930D66D90 in vertex: E6E183917E994AF98262AB6649D7AF69
2016-01-14 17:16:38,602 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: E6E183917E994AF98262AB6649D7AF69 with 1 source tasks and 1 pending tasks
2016-01-14 17:16:38,602 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:38,602 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791798427_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791798427_0001_1_01, startRequestedTime=1452791798602, startedTime=1452791798602
2016-01-14 17:16:38,603 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791798427_0001_1_01 [E6E183917E994AF98262AB6649D7AF69] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:38,603 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [0FFF661DC8274DA79B10D3BAFD3990C2/B290CE9CF7D246E2B14BC587B5CE2C8B] every/(1/1) ...emblyplatform/0_VALUES_-1, VertexName: A620AFF2E7154A089412E06930D66D90, VertexParallelism: 1, TaskAttemptID:attempt_1452791798427_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EDE301EC31844B4A97E06348C7C8E8AE, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E6E183917E994AF98262AB6649D7AF69, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:38,603 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791798427_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:38,603 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:38,604 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:38,604 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:38,604 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:38,604 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791798427_0001_00_000001
2016-01-14 17:16:38,604 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791798427_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:38,606 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791798427_0001_00_000001
2016-01-14 17:16:38,606 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:38,607 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791798427_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:38,607 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:38,607 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791798427_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791798427_0001_00_000001, launchTime=1452791798607
2016-01-14 17:16:38,608 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791798427_0001_00_000001
2016-01-14 17:16:38,608 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791798427_0001_1_00_000000_0] to container: [container_1452791798427_0001_00_000001]
2016-01-14 17:16:38,608 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791798427_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:38,609 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791798427_0001_00_000001 given task: attempt_1452791798427_0001_1_00_000000_0
2016-01-14 17:16:38,609 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791798427_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791798427_0001_1_00_000000_0
2016-01-14 17:16:38,609 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791798427_0001_1_00_000000_0] started. Is using containerId: [container_1452791798427_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:38,609 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791798427_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A620AFF2E7154A089412E06930D66D90, taskAttemptId=attempt_1452791798427_0001_1_00_000000_0, startTime=1452791798609, containerId=container_1452791798427_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791798427_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:38,609 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791798427_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:38,609 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791798427_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:38 Starting to run new task attempt: attempt_1452791798427_0001_1_00_000000_0
2016-01-14 17:16:38,609 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:38,610 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791798427_0001_1_00_000000_0
2016-01-14 17:16:38,610 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:38,610 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:38,610 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:38,610 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:38,610 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:38,610 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:38,611 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [0FFF661DC8274DA79B10D3BAFD3990C2/B290CE9CF7D246E2B14BC587B5CE2C8B] every/(1/1) ...emblyplatform/0_VALUES_-1, VertexName: A620AFF2E7154A089412E06930D66D90, VertexParallelism: 1, TaskAttemptID:attempt_1452791798427_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EDE301EC31844B4A97E06348C7C8E8AE, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E6E183917E994AF98262AB6649D7AF69, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:38,611 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=751828992
2016-01-14 17:16:38,611 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791798427_0001_1_00_000000_0
2016-01-14 17:16:38,611 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:38,611 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:38,612 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791798427_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:38,612 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=EDE301EC31844B4A97E06348C7C8E8AE, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:38,612 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:38,613 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: EDE301EC31844B4A97E06348C7C8E8AE
2016-01-14 17:16:38,613 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: EDE301EC31844B4A97E06348C7C8E8AE
2016-01-14 17:16:38,613 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:38,614 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=E6E183917E994AF98262AB6649D7AF69, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:38,614 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:38,614 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: E6E183917E994AF98262AB6649D7AF69
2016-01-14 17:16:38,615 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791798427_0001, with dag id: dag_1452791798427_0001_1
2016-01-14 17:16:38,616 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:38,617 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:38,617 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: E6E183917E994AF98262AB6649D7AF69
2016-01-14 17:16:38,617 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: E6E183917E994AF98262AB6649D7AF69
2016-01-14 17:16:38,637 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:38,637 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: EDE301EC31844B4A97E06348C7C8E8AE
2016-01-14 17:16:38,637 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: EDE301EC31844B4A97E06348C7C8E8AE
2016-01-14 17:16:38,641 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A620AFF2E7154A089412E06930D66D90, ordinal: 0
2016-01-14 17:16:38,641 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A620AFF2E7154A089412E06930D66D90, mem on start (mb), free: 466, total: 711, max: 711
2016-01-14 17:16:38,641 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:38,641 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:38,641 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:38,641 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:38,641 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:38,641 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:38,642 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:38,642 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:38,642 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:38,642 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 745537536, TotalAvailable: 503725424, TotalRequested/TotalJVMHeap:0.14
2016-01-14 17:16:38,642 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, EDE301EC31844B4A97E06348C7C8E8AE, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:38,642 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, E6E183917E994AF98262AB6649D7AF69, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:38,642 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:38,643 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: EDE301EC31844B4A97E06348C7C8E8AE
2016-01-14 17:16:38,643 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:38,643 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: EDE301EC31844B4A97E06348C7C8E8AE
2016-01-14 17:16:38,643 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: EDE301EC31844B4A97E06348C7C8E8AE being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:38,643 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:38,644 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:38,644 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:38,644 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791798427_0001_1_00_000000_0
2016-01-14 17:16:38,644 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:38,645 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@55fd571a
2016-01-14 17:16:38,645 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@55fd571a
2016-01-14 17:16:38,645 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:38,645 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:38,647 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:38,648 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: EDE301EC31844B4A97E06348C7C8E8AE
2016-01-14 17:16:38,648 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 19EF329CEFE841F29435CD6915F5D55D
2016-01-14 17:16:38,648 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 19EF329CEFE841F29435CD6915F5D55D
2016-01-14 17:16:38,649 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@28632707; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@5975148; comparator=cascading.tuple.hadoop.util.TupleComparator@115950d6; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:38,649 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:38,649 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:38,649 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:38,662 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:38,663 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:38,663 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:38,663 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:38,663 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:38,663 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:38,664 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:38,664 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A620AFF2E7154A089412E06930D66D90, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:38,667 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791798427_0001_1_00_000000_0
2016-01-14 17:16:38,667 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A620AFF2E7154A089412E06930D66D90, mem on close (mb), free: 365, total: 711, max: 711
2016-01-14 17:16:38,668 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:38,668 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:38,668 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:38,668 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:38,671 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:38 Completed running task attempt: attempt_1452791798427_0001_1_00_000000_0
2016-01-14 17:16:38,673 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791798427_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:38,674 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791798427_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791798427_0001_1_00, eventCount=1
2016-01-14 17:16:38,674 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=18, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=745537536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791798664, Process_Duration=3, Process_End_Time=1452791798667, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:38,674 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:38,675 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:38,675 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:38,675 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791798427_0001_00_000001
2016-01-14 17:16:38,674 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791798427_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A620AFF2E7154A089412E06930D66D90, taskAttemptId=attempt_1452791798427_0001_1_00_000000_0, startTime=1452791798609, finishTime=1452791798674, timeTaken=65, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=18, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=745537536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791798664, Process_Duration=3, Process_End_Time=1452791798667, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:38,675 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791798427_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:38,675 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791798427_0001_1_00_000000_0
2016-01-14 17:16:38,675 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:38,676 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:38,676 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791798427_0001_1][Event:TASK_FINISHED]: vertexName=A620AFF2E7154A089412E06930D66D90, taskId=task_1452791798427_0001_1_00_000000, startTime=1452791798609, finishTime=1452791798675, timeTaken=66, status=SUCCEEDED, successfulAttemptID=attempt_1452791798427_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=18, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=745537536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791798664, Process_Duration=3, Process_End_Time=1452791798667, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:38,676 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791798427_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:38,676 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791798427_0001_1_00 [A620AFF2E7154A089412E06930D66D90] : 1
2016-01-14 17:16:38,676 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791798427_0001_1_00 [A620AFF2E7154A089412E06930D66D90]
2016-01-14 17:16:38,677 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:38,677 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791798427_0001_1][Event:VERTEX_FINISHED]: vertexName=A620AFF2E7154A089412E06930D66D90, vertexId=vertex_1452791798427_0001_1_00, initRequestedTime=1452791798590, initedTime=1452791798599, startRequestedTime=1452791798599, startedTime=1452791798599, finishTime=1452791798676, timeTaken=77, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=18, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=745537536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791798664, Process_Duration=3, Process_End_Time=1452791798667, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791798609, firstTasksToStart=[ task_1452791798427_0001_1_00_000000 ], lastTaskFinishTime=1452791798674, lastTasksToFinish=[ task_1452791798427_0001_1_00_000000 ], minTaskDuration=65, maxTaskDuration=65, avgTaskDuration=65.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791798427_0001_1_00_000000 ], longestDurationTasks=[ task_1452791798427_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:38,677 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791798427_0001_1_00 [A620AFF2E7154A089412E06930D66D90] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:38,677 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791798427_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:38,677 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791798427_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791798427_0001_00_000001, stoppedTime=1452791798677, exitStatus=0
2016-01-14 17:16:38,677 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791798427_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:38,677 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791798427_0001_1_01 [E6E183917E994AF98262AB6649D7AF69] attempt: attempt_1452791798427_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:38,678 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: E6E183917E994AF98262AB6649D7AF69
2016-01-14 17:16:38,678 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791798427_0001_1_01 [E6E183917E994AF98262AB6649D7AF69]
2016-01-14 17:16:38,678 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791798427_0001_1_00 [A620AFF2E7154A089412E06930D66D90] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:38,678 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791798427_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:38,677 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:38,679 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791798427_0001_1][Event:TASK_STARTED]: vertexName=E6E183917E994AF98262AB6649D7AF69, taskId=task_1452791798427_0001_1_01_000000, scheduledTime=1452791798678, launchTime=1452791798678
2016-01-14 17:16:38,679 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791798427_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:38,679 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:38,679 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791798427_0001_1_01 [E6E183917E994AF98262AB6649D7AF69]
2016-01-14 17:16:38,679 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791798427_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:38,679 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [0FFF661DC8274DA79B10D3BAFD3990C2/B290CE9CF7D246E2B14BC587B5CE2C8B] every/(1/1) ...emblyplatform/0_VALUES_-1, VertexName: E6E183917E994AF98262AB6649D7AF69, VertexParallelism: 1, TaskAttemptID:attempt_1452791798427_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A620AFF2E7154A089412E06930D66D90, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=36C44688EBA544D7A931ED9BDF10799C, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:38,679 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791798427_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:38,679 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:38,680 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791798427_0001_00_000002
2016-01-14 17:16:38,680 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791798427_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:38,682 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791798427_0001_00_000002
2016-01-14 17:16:38,682 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:38,682 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791798427_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:38,682 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791798427_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791798427_0001_00_000002, launchTime=1452791798682
2016-01-14 17:16:38,682 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:38,683 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791798427_0001_00_000002
2016-01-14 17:16:38,683 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791798427_0001_1_01_000000_0] to container: [container_1452791798427_0001_00_000002]
2016-01-14 17:16:38,683 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791798427_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:38,684 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791798427_0001_1_01_000000_0] started. Is using containerId: [container_1452791798427_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:38,684 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791798427_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E6E183917E994AF98262AB6649D7AF69, taskAttemptId=attempt_1452791798427_0001_1_01_000000_0, startTime=1452791798684, containerId=container_1452791798427_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791798427_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:38,684 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:38,684 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791798427_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:38,684 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791798427_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:38,684 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791798427_0001_00_000002 given task: attempt_1452791798427_0001_1_01_000000_0
2016-01-14 17:16:38,684 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791798427_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791798427_0001_1_01_000000_0
2016-01-14 17:16:38 Starting to run new task attempt: attempt_1452791798427_0001_1_01_000000_0
2016-01-14 17:16:38,685 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791798427_0001_1_01_000000_0
2016-01-14 17:16:38,685 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:38,685 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:38,685 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:38,685 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:38,685 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:38,685 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:38,685 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [0FFF661DC8274DA79B10D3BAFD3990C2/B290CE9CF7D246E2B14BC587B5CE2C8B] every/(1/1) ...emblyplatform/0_VALUES_-1, VertexName: E6E183917E994AF98262AB6649D7AF69, VertexParallelism: 1, TaskAttemptID:attempt_1452791798427_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A620AFF2E7154A089412E06930D66D90, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=36C44688EBA544D7A931ED9BDF10799C, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:38,686 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=745537536
2016-01-14 17:16:38,686 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791798427_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:38,686 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791798427_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:38,686 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791798427_0001_00_000001, Exiting
2016-01-14 17:16:38,686 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791798427_0001_00_000001
2016-01-14 17:16:38,686 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791798427_0001_00_000001
2016-01-14 17:16:38,687 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791798427_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:38,687 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791798427_0001_1_01_000000_0
2016-01-14 17:16:38,687 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:38,687 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:38,689 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:38,689 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=A620AFF2E7154A089412E06930D66D90, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:38,689 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791798427_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:38,689 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:38,690 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: A620AFF2E7154A089412E06930D66D90
2016-01-14 17:16:38,692 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:38,693 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 670983744, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:38,693 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:38,693 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 670983744, type: INPUT, componentVertexName: A620AFF2E7154A089412E06930D66D90
2016-01-14 17:16:38,693 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: A620AFF2E7154A089412E06930D66D90
2016-01-14 17:16:38,696 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=36C44688EBA544D7A931ED9BDF10799C, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:38,696 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:38,697 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791798427_0001_00_000001 completed successfully
2016-01-14 17:16:38,697 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791798427_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:38,697 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 36C44688EBA544D7A931ED9BDF10799C
2016-01-14 17:16:38,697 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:38,698 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 36C44688EBA544D7A931ED9BDF10799C
2016-01-14 17:16:38,700 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E6E183917E994AF98262AB6649D7AF69, ordinal: 1
2016-01-14 17:16:38,700 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E6E183917E994AF98262AB6649D7AF69, mem on start (mb), free: 359, total: 711, max: 711
2016-01-14 17:16:38,701 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:38,701 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:38,701 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:38,709 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:38,709 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 36C44688EBA544D7A931ED9BDF10799C
2016-01-14 17:16:38,709 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:38,709 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:38,709 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:38,709 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:38,710 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:38,710 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:38,710 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 670983744, TotalRequestedScaled: 6.193696098461539E8, TotalJVMHeap: 745537536, TotalAvailable: 499510149, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:38,710 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, A620AFF2E7154A089412E06930D66D90, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=670983744, allocated=499510149
2016-01-14 17:16:38,710 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 36C44688EBA544D7A931ED9BDF10799C, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:38,710 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:38,710 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: A620AFF2E7154A089412E06930D66D90 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:38,710 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:38,710 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:38,710 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: A620AFF2E7154A089412E06930D66D90
2016-01-14 17:16:38,711 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:38,711 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:38,711 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: A620AFF2E7154A089412E06930D66D90 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:38,712 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=670983744, postMergeMem=0, RuntimeTotalAvailable=499510149. Updated to: ShuffleMem=499510149, postMergeMem: 0
2016-01-14 17:16:38,712 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=499510149, maxSingleShuffleLimit=124877536, mergeThreshold=449559104, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:38,712 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:38,712 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: A620AFF2E7154A089412E06930D66D90
2016-01-14 17:16:38,713 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:38,713 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791798427_0001_1_01_000000_0
2016-01-14 17:16:38,718 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:38,718 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791798427_0001_1_00_000000_0_10177, runDuration: 57572, ]
2016-01-14 17:16:38,719 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 19EF329CEFE841F29435CD6915F5D55D
2016-01-14 17:16:38,719 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_-1"], id: 36C44688EBA544D7A931ED9BDF10799C
2016-01-14 17:16:38,719 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_-1"]
2016-01-14 17:16:38,719 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 19EF329CEFE841F29435CD6915F5D55D, for 1 inputs
2016-01-14 17:16:38,721 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791798427_0001_1_00_000000_0_10177] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791798721, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 17:16:38,721 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : A620AFF2E7154A089412E06930D66D90
2016-01-14 17:16:38,721 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:38,721 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [A620AFF2E7154A089412E06930D66D90] #30 in 1ms
2016-01-14 17:16:38,722 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [A620AFF2E7154A089412E06930D66D90] #30, status:false, isInterrupted:false
2016-01-14 17:16:38,722 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:38,723 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:38,723 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:38,723 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:38,723 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:38,723 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : A620AFF2E7154A089412E06930D66D90
2016-01-14 17:16:38,723 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:38,723 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E6E183917E994AF98262AB6649D7AF69, all 1 inputs ready in: 00:00:00.004
2016-01-14 17:16:38,724 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@3e7d948b; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@26a351ba
2016-01-14 17:16:38,725 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:38,725 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(92)) - caught throwable
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:38,725 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:38,725 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:38,726 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791798427_0001_1_01_000000_0
2016-01-14 17:16:38,726 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791798427_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:38,726 INFO  task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1452791798427_0001_1_01_000000_0
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:38,727 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=745537536, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=9, LAST_EVENT_RECEIVED=9][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791798723, Process_Duration=3, Process_End_Time=1452791798726, Read_Duration=0, Tuples_Read=1]]
2016-01-14 17:16:38,727 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E6E183917E994AF98262AB6649D7AF69, mem on close (mb), free: 356, total: 711, max: 711
2016-01-14 17:16:38,727 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=E6E183917E994AF98262AB6649D7AF69, index=1
2016-01-14 17:16:38,727 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791798427_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E6E183917E994AF98262AB6649D7AF69, taskAttemptId=attempt_1452791798427_0001_1_01_000000_0, startTime=1452791798684, finishTime=1452791798727, timeTaken=43, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=745537536, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=9, LAST_EVENT_RECEIVED=9, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791798723, Process_Duration=3, Process_End_Time=1452791798726, Read_Duration=0, Tuples_Read=1
2016-01-14 17:16:38,727 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: A620AFF2E7154A089412E06930D66D90
2016-01-14 17:16:38,728 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791798427_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-01-14 17:16:38,728 INFO  impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1452791798427_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-01-14 17:16:38,728 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791798427_0001_1][Event:TASK_FINISHED]: vertexName=E6E183917E994AF98262AB6649D7AF69, taskId=task_1452791798427_0001_1_01_000000, startTime=1452791798684, finishTime=1452791798728, timeTaken=44, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
], counters=Counters: 0
2016-01-14 17:16:38,728 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791798427_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-01-14 17:16:38,728 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:38,728 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:38,728 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791798427_0001_1_01 [E6E183917E994AF98262AB6649D7AF69] : 1
2016-01-14 17:16:38,729 INFO  impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1452791798427_0001_1_01 [E6E183917E994AF98262AB6649D7AF69] because task failed: task_1452791798427_0001_1_01_000000
2016-01-14 17:16:38,729 INFO  impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1452791798427_0001_1_01 [E6E183917E994AF98262AB6649D7AF69] due to trigger: OWN_TASK_FAILURE
2016-01-14 17:16:38,729 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-01-14 17:16:38,729 INFO  impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1452791798427_0001_1_01 [E6E183917E994AF98262AB6649D7AF69]
2016-01-14 17:16:38,729 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:38,730 INFO  impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=36C44688EBA544D7A931ED9BDF10799C, vertexId=vertex_1452791798427_0001_1_01 [E6E183917E994AF98262AB6649D7AF69]
2016-01-14 17:16:38,730 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:38,731 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:38,731 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:38,731 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:38,731 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:38,731 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=E6E183917E994AF98262AB6649D7AF69, sourceVertex=A620AFF2E7154A089412E06930D66D90
2016-01-14 17:16:38,731 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=E6E183917E994AF98262AB6649D7AF69, sourceVertex=36C44688EBA544D7A931ED9BDF10799C
2016-01-14 17:16:38,731 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791798427_0001_00_000002
2016-01-14 17:16:38,732 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791798427_0001_00_000002
2016-01-14 17:16:38,732 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791798427_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-01-14 17:16:38,732 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1452791798427_0001_00_000002 completed but with errors
2016-01-14 17:16:38,732 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791798427_0001_1][Event:VERTEX_FINISHED]: vertexName=E6E183917E994AF98262AB6649D7AF69, vertexId=vertex_1452791798427_0001_1_01, initRequestedTime=1452791798591, initedTime=1452791798597, startRequestedTime=1452791798602, startedTime=1452791798602, finishTime=1452791798732, timeTaken=130, status=FAILED, diagnostics=Task failed, taskId=task_1452791798427_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1452791798427_0001_1_01 [E6E183917E994AF98262AB6649D7AF69] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1452791798684, firstTasksToStart=[ task_1452791798427_0001_1_01_000000 ], lastTaskFinishTime=1452791798727, lastTasksToFinish=[ task_1452791798427_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[  ], longestDurationTasks=[  ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-01-14 17:16:38,732 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:38,734 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791798427_0001_1_01 [E6E183917E994AF98262AB6649D7AF69] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-01-14 17:16:38,735 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791798427_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791798427_0001_00_000002, stoppedTime=1452791798735, exitStatus=0
2016-01-14 17:16:38,735 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791798427_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:38,735 INFO  node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1452791798427_0001_1_01_000000_0 failed: true container: container_1452791798427_0001_00_000002 numFailedTAs: 0
2016-01-14 17:16:38,736 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791798427_0001_1_01 [E6E183917E994AF98262AB6649D7AF69] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-01-14 17:16:38,735 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:38,736 INFO  impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-01-14 17:16:38,736 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791798427_0001_1
2016-01-14 17:16:38,737 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1452791798427_0001_1_00 [A620AFF2E7154A089412E06930D66D90]
2016-01-14 17:16:38,737 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 36C44688EBA544D7A931ED9BDF10799C for vertex: vertex_1452791798427_0001_1_01 [E6E183917E994AF98262AB6649D7AF69]
2016-01-14 17:16:38,738 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791798427_0001_1, queueSize=0
2016-01-14 17:16:38,739 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:38,740 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791798427_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791798427_0001_1, startTime=1452791798589, finishTime=1452791798737, timeTaken=148, status=FAILED, diagnostics=Vertex failed, vertexName=E6E183917E994AF98262AB6649D7AF69, vertexId=vertex_1452791798427_0001_1_01, diagnostics=[Task failed, taskId=task_1452791798427_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1452791798427_0001_1_01 [E6E183917E994AF98262AB6649D7AF69] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=18, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=745537536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791798664, Process_Duration=3, Process_End_Time=1452791798667, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:38,740 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 17:16:38,742 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 17:16:38,742 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791798427_0001_1 finished with state: FAILED
2016-01-14 17:16:38,743 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791798427_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:38,743 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1452791798427_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-01-14 17:16:38,743 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791798427_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:16:38,743 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452791798427_0001_00_000002
2016-01-14 17:16:38 Completed Dag: dag_1452791798427_0001_1
2016-01-14 17:16:38,744 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791798427_0001_1, dagState=FAILED
2016-01-14 17:16:38,744 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791798427_0001_1_post
2016-01-14 17:16:38,744 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:38,745 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:38,745 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791798427_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-01-14 17:16:38,752 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...emblyplatform/0_VALUES_-1, applicationId=application_1452791798427_0001
2016-01-14 17:16:38,753 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:38,753 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:38,753 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:38,753 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:38,753 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:38,754 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:38,754 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:38,754 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:38,755 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:38,755 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:38,755 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:38,755 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:38,755 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:38,760 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/0FFF661DC8274DA79B10D3BAFD3990C2/B290CE9CF7D246E2B14BC587B5CE2C8B/.tez/application_1452791798427_0001
2016-01-14 17:16:38,760 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping all jobs
2016-01-14 17:16:38,760 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping: (1/1) ...emblyplatform/0_VALUES_-1
2016-01-14 17:16:38,761 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopped all jobs
2016-01-14 17:16:38,761 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.351
2016-01-14 17:16:38,761 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_VALUES_-1
2016-01-14 17:16:38,782 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:38,783 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:38,806 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.002
2016-01-14 17:16:38,809 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.005
2016-01-14 17:16:38,809 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 17:16:38,809 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 17:16:38,809 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_VALUES_'field'
2016-01-14 17:16:38,828 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:38,828 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:38,857 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.008
2016-01-14 17:16:38,864 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-01-14 17:16:38,864 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:38,865 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:38,865 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:38,878 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:38,884 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_RESULTS
2016-01-14 17:16:38,926 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:38,926 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:38,926 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_RESULTS"]
2016-01-14 17:16:38,926 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:38,926 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:38,926 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:38,928 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...platform/0_VALUES_RESULTS
2016-01-14 17:16:38,928 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:38,939 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:38,944 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:38,944 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:38,944 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:38,954 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A61B784B8FD24706A1FC8E022F4F10F2/BE726C1C752F420FB26FA3C74B6505F5/.tez/application_1452791798944_0001 doesn't exist and is created
2016-01-14 17:16:38,970 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:38,970 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:38,970 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A61B784B8FD24706A1FC8E022F4F10F2/BE726C1C752F420FB26FA3C74B6505F5/.tez/application_1452791798944_0001
2016-01-14 17:16:38,973 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791798944_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:38,982 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:38,983 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:38,984 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:38,985 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:38,985 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A61B784B8FD24706A1FC8E022F4F10F2/BE726C1C752F420FB26FA3C74B6505F5/.tez/application_1452791798944_0001/localmode-log-dir
2016-01-14 17:16:38,985 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A61B784B8FD24706A1FC8E022F4F10F2/BE726C1C752F420FB26FA3C74B6505F5/.tez/application_1452791798944_0001/localmode-log-dir/history.txt.appattempt_1452791798944_0001_000000, maxErrors=10
2016-01-14 17:16:38,985 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:38,986 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791798944_0001_000000, appSubmitTime=1452791798972, launchTime=1452791798972
2016-01-14 17:16:38,990 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:39460
2016-01-14 17:16:38,990 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:38,993 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:38,994 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:38,995 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:38,996 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:38,997 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791798944_0001_000000, startTime=1452791798997
2016-01-14 17:16:38,997 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:38,998 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:39,070 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:39,070 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:39,070 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...platform/0_VALUES_RESULTS, applicationId=application_1452791798944_0001, dagName=[A61B784B8FD24706A1FC8E022F4F10F2/BE726C1C752F420FB26FA3C74B6505F5] every/(1/1) ...platform/0_VALUES_RESULTS
2016-01-14 17:16:39,085 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [A61B784B8FD24706A1FC8E022F4F10F2/BE726C1C752F420FB26FA3C74B6505F5] every/(1/1) ...platform/0_VALUES_RESULTS
2016-01-14 17:16:39,086 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791798944_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A61B784B8FD24706A1FC8E022F4F10F2/BE726C1C752F420FB26FA3C74B6505F5/.tez/application_1452791798944_0001/localmode-log-dir/dag_1452791798944_0001_1.dot
2016-01-14 17:16:39,087 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A61B784B8FD24706A1FC8E022F4F10F2/BE726C1C752F420FB26FA3C74B6505F5/.tez/application_1452791798944_0001/localmode-log-dir/dag_1452791798944_0001_1-tez-dag.pb.txt
2016-01-14 17:16:39,092 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791798944_0001_1
2016-01-14 17:16:39,092 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:39,093 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [A61B784B8FD24706A1FC8E022F4F10F2/BE726C1C752F420FB26FA3C74B6505F5] every/(1/1) ...platform/0_VALUES_RESULTS
2016-01-14 17:16:39 Running Dag: dag_1452791798944_0001_1
2016-01-14 17:16:39,104 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:39,104 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791798944_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791798944_0001_1, submitTime=1452791799085
2016-01-14 17:16:39,105 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex CA227A3CEC4D44AF8E6A8D6AFA5562BB
2016-01-14 17:16:39,105 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:39,106 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:39,106 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791798944_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791798944_0001_1, initTime=1452791799104
2016-01-14 17:16:39,106 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791798944_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:39,107 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791798944_0001_1][Event:DAG_STARTED]: dagID=dag_1452791798944_0001_1, startTime=1452791799107
2016-01-14 17:16:39,107 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:39,107 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791798944_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:39,108 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: B0211B94D940448DABF0062F352EBFBF : {01C4118E8AA2413AAF301671D751BA1B={InputName=01C4118E8AA2413AAF301671D751BA1B}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:39,108 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 01C4118E8AA2413AAF301671D751BA1B, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:39,108 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791798944_0001_1_00 [B0211B94D940448DABF0062F352EBFBF]
2016-01-14 17:16:39,108 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791798944_0001_1_00 [B0211B94D940448DABF0062F352EBFBF]
2016-01-14 17:16:39,108 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791798944_0001_1_00 [B0211B94D940448DABF0062F352EBFBF]
2016-01-14 17:16:39,108 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791798944_0001_1_00 [B0211B94D940448DABF0062F352EBFBF]. Starting root input initializers: 1
2016-01-14 17:16:39,107 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:39,108 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791798944_0001_1_00 [B0211B94D940448DABF0062F352EBFBF] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:39,108 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791798944_0001_1_01 [CA227A3CEC4D44AF8E6A8D6AFA5562BB]
2016-01-14 17:16:39,108 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...platform/0_VALUES_RESULTS, applicationId=application_1452791798944_0001, dagName=[A61B784B8FD24706A1FC8E022F4F10F2/BE726C1C752F420FB26FA3C74B6505F5] every/(1/1) ...platform/0_VALUES_RESULTS
2016-01-14 17:16:39,111 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 01C4118E8AA2413AAF301671D751BA1B on vertex vertex_1452791798944_0001_1_00 [B0211B94D940448DABF0062F352EBFBF]
2016-01-14 17:16:39,111 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:39,111 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791798944_0001_1_01 [CA227A3CEC4D44AF8E6A8D6AFA5562BB]
2016-01-14 17:16:39,112 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791798944_0001_1_01 [CA227A3CEC4D44AF8E6A8D6AFA5562BB]
2016-01-14 17:16:39,112 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791798944_0001_1_01 [CA227A3CEC4D44AF8E6A8D6AFA5562BB]
2016-01-14 17:16:39,112 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=FBC0BCE68E334493BB905AB510537591, vertexId=vertex_1452791798944_0001_1_01 [CA227A3CEC4D44AF8E6A8D6AFA5562BB], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:39,112 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=FBC0BCE68E334493BB905AB510537591, vertexId=vertex_1452791798944_0001_1_01 [CA227A3CEC4D44AF8E6A8D6AFA5562BB]
2016-01-14 17:16:39,114 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=FBC0BCE68E334493BB905AB510537591, vertexName=CA227A3CEC4D44AF8E6A8D6AFA5562BB, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:39,114 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:39,114 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=FBC0BCE68E334493BB905AB510537591, vertexId=vertex_1452791798944_0001_1_01 [CA227A3CEC4D44AF8E6A8D6AFA5562BB]
2016-01-14 17:16:39,115 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 01C4118E8AA2413AAF301671D751BA1B asking for 1 tasks. Headroom: 711 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:39,115 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:39,115 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:39,116 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:39,116 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791798944_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CA227A3CEC4D44AF8E6A8D6AFA5562BB, vertexId=vertex_1452791798944_0001_1_01, initRequestedTime=1452791799108, initedTime=1452791799116, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:39,117 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791798944_0001_1_01 [CA227A3CEC4D44AF8E6A8D6AFA5562BB] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:39,117 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:39,117 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 01C4118E8AA2413AAF301671D751BA1B on vertex vertex_1452791798944_0001_1_00 [B0211B94D940448DABF0062F352EBFBF]
2016-01-14 17:16:39,118 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791798944_0001_1_00 [B0211B94D940448DABF0062F352EBFBF] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:39,118 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {01C4118E8AA2413AAF301671D751BA1B=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:39,118 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791798944_0001_1_00 [B0211B94D940448DABF0062F352EBFBF] parallelism set to 1
2016-01-14 17:16:39,118 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791798944_0001_1_00 [B0211B94D940448DABF0062F352EBFBF] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:39,119 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791798944_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791798944_0001_1_00, eventCount=1
2016-01-14 17:16:39,119 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791798944_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B0211B94D940448DABF0062F352EBFBF, vertexId=vertex_1452791798944_0001_1_00, initRequestedTime=1452791799107, initedTime=1452791799119, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:39,119 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791798944_0001_1_00 [B0211B94D940448DABF0062F352EBFBF] with distanceFromRoot: 0
2016-01-14 17:16:39,119 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:39,120 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791798944_0001_1_00 [B0211B94D940448DABF0062F352EBFBF] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:39,120 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in B0211B94D940448DABF0062F352EBFBF
2016-01-14 17:16:39,120 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791798944_0001_1_00 [B0211B94D940448DABF0062F352EBFBF]
2016-01-14 17:16:39,120 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791798944_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791798944_0001_1_00, startRequestedTime=1452791799120, startedTime=1452791799120
2016-01-14 17:16:39,120 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:39,121 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791798944_0001_1_00 [B0211B94D940448DABF0062F352EBFBF] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:39,121 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:39,121 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791798944_0001_1][Event:TASK_STARTED]: vertexName=B0211B94D940448DABF0062F352EBFBF, taskId=task_1452791798944_0001_1_00_000000, scheduledTime=1452791799121, launchTime=1452791799121
2016-01-14 17:16:39,121 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791798944_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:39,122 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791798944_0001_1_00 for vertex: vertex_1452791798944_0001_1_01 [CA227A3CEC4D44AF8E6A8D6AFA5562BB] numStartedSources: 1 numSources: 1
2016-01-14 17:16:39,122 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791798944_0001_1_01 [CA227A3CEC4D44AF8E6A8D6AFA5562BB] with distanceFromRoot: 1
2016-01-14 17:16:39,122 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791798944_0001_1_00 [B0211B94D940448DABF0062F352EBFBF]
2016-01-14 17:16:39,122 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: B0211B94D940448DABF0062F352EBFBF in vertex: CA227A3CEC4D44AF8E6A8D6AFA5562BB
2016-01-14 17:16:39,122 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: CA227A3CEC4D44AF8E6A8D6AFA5562BB with 1 source tasks and 1 pending tasks
2016-01-14 17:16:39,123 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791798944_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791798944_0001_1_01, startRequestedTime=1452791799122, startedTime=1452791799122
2016-01-14 17:16:39,123 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:39,123 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791798944_0001_1_01 [CA227A3CEC4D44AF8E6A8D6AFA5562BB] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:39,123 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [A61B784B8FD24706A1FC8E022F4F10F2/BE726C1C752F420FB26FA3C74B6505F5] every/(1/1) ...platform/0_VALUES_RESULTS, VertexName: B0211B94D940448DABF0062F352EBFBF, VertexParallelism: 1, TaskAttemptID:attempt_1452791798944_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=01C4118E8AA2413AAF301671D751BA1B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CA227A3CEC4D44AF8E6A8D6AFA5562BB, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:39,123 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791798944_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:39,124 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:39,124 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:39,125 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:39,125 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:39,125 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791798944_0001_00_000001
2016-01-14 17:16:39,125 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791798944_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:39,126 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791798944_0001, with dag id: dag_1452791798944_0001_1
2016-01-14 17:16:39,127 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791798944_0001_00_000001
2016-01-14 17:16:39,127 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:39,128 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791798944_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:39,128 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:39,128 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791798944_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791798944_0001_00_000001, launchTime=1452791799128
2016-01-14 17:16:39,129 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791798944_0001_00_000001
2016-01-14 17:16:39,130 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791798944_0001_1_00_000000_0] to container: [container_1452791798944_0001_00_000001]
2016-01-14 17:16:39,130 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791798944_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:39,131 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791798944_0001_1_00_000000_0] started. Is using containerId: [container_1452791798944_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:39,131 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791798944_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B0211B94D940448DABF0062F352EBFBF, taskAttemptId=attempt_1452791798944_0001_1_00_000000_0, startTime=1452791799131, containerId=container_1452791798944_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791798944_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:39,132 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791798944_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:39,132 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:39,132 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791798944_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:39,132 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791798944_0001_00_000001 given task: attempt_1452791798944_0001_1_00_000000_0
2016-01-14 17:16:39,132 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791798944_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791798944_0001_1_00_000000_0
2016-01-14 17:16:39 Starting to run new task attempt: attempt_1452791798944_0001_1_00_000000_0
2016-01-14 17:16:39,132 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791798944_0001_1_00_000000_0
2016-01-14 17:16:39,133 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:39,133 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:39,133 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:39,133 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:39,133 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:39,134 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:39,134 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [A61B784B8FD24706A1FC8E022F4F10F2/BE726C1C752F420FB26FA3C74B6505F5] every/(1/1) ...platform/0_VALUES_RESULTS, VertexName: B0211B94D940448DABF0062F352EBFBF, VertexParallelism: 1, TaskAttemptID:attempt_1452791798944_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=01C4118E8AA2413AAF301671D751BA1B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CA227A3CEC4D44AF8E6A8D6AFA5562BB, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:39,134 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=745537536
2016-01-14 17:16:39,134 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791798944_0001_1_00_000000_0
2016-01-14 17:16:39,134 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:39,135 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:39,135 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791798944_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:39,135 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:39,135 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=01C4118E8AA2413AAF301671D751BA1B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:39,135 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=CA227A3CEC4D44AF8E6A8D6AFA5562BB, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:39,136 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:39,136 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:39,136 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: CA227A3CEC4D44AF8E6A8D6AFA5562BB
2016-01-14 17:16:39,136 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 01C4118E8AA2413AAF301671D751BA1B
2016-01-14 17:16:39,137 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 01C4118E8AA2413AAF301671D751BA1B
2016-01-14 17:16:39,138 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:39,138 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: CA227A3CEC4D44AF8E6A8D6AFA5562BB
2016-01-14 17:16:39,156 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: CA227A3CEC4D44AF8E6A8D6AFA5562BB
2016-01-14 17:16:39,157 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:39,157 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 01C4118E8AA2413AAF301671D751BA1B
2016-01-14 17:16:39,157 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 01C4118E8AA2413AAF301671D751BA1B
2016-01-14 17:16:39,159 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:39,163 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B0211B94D940448DABF0062F352EBFBF, ordinal: 0
2016-01-14 17:16:39,163 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B0211B94D940448DABF0062F352EBFBF, mem on start (mb), free: 472, total: 717, max: 717
2016-01-14 17:16:39,163 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:39,163 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:39,163 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:39,163 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:39,163 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:39,163 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:39,164 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:39,164 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:39,164 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:39,164 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 751828992, TotalAvailable: 499510149, TotalRequested/TotalJVMHeap:0.14
2016-01-14 17:16:39,164 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 01C4118E8AA2413AAF301671D751BA1B, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:39,164 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, CA227A3CEC4D44AF8E6A8D6AFA5562BB, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:39,165 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:39,165 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 01C4118E8AA2413AAF301671D751BA1B
2016-01-14 17:16:39,165 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:39,165 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 01C4118E8AA2413AAF301671D751BA1B
2016-01-14 17:16:39,165 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 01C4118E8AA2413AAF301671D751BA1B being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:39,166 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:39,166 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:39,166 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:39,166 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791798944_0001_1_00_000000_0
2016-01-14 17:16:39,166 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:39,167 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@387f4217
2016-01-14 17:16:39,167 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@387f4217
2016-01-14 17:16:39,168 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:39,168 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:39,170 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:39,171 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 01C4118E8AA2413AAF301671D751BA1B
2016-01-14 17:16:39,172 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 3E9D99564E6843F0A58030B96B48A615
2016-01-14 17:16:39,172 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 3E9D99564E6843F0A58030B96B48A615
2016-01-14 17:16:39,172 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@243ac9a0; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@f8b39f5; comparator=cascading.tuple.hadoop.util.TupleComparator@7d8a0bcb; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:39,173 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:39,173 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:39,173 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:39,187 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:39,187 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:39,187 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:39,188 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:39,188 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:39,188 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:39,189 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:39,189 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B0211B94D940448DABF0062F352EBFBF, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:39,191 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791798944_0001_1_00_000000_0
2016-01-14 17:16:39,192 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B0211B94D940448DABF0062F352EBFBF, mem on close (mb), free: 370, total: 717, max: 717
2016-01-14 17:16:39,192 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:39,192 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:39,192 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:39,192 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:39,194 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:39 Completed running task attempt: attempt_1452791798944_0001_1_00_000000_0
2016-01-14 17:16:39,198 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791798944_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:39,199 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=18, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751828992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791799189, Process_Duration=2, Process_End_Time=1452791799191, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:39,199 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791798944_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791798944_0001_1_00, eventCount=1
2016-01-14 17:16:39,199 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:39,199 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:39,199 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:39,199 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791798944_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B0211B94D940448DABF0062F352EBFBF, taskAttemptId=attempt_1452791798944_0001_1_00_000000_0, startTime=1452791799131, finishTime=1452791799199, timeTaken=68, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=18, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751828992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791799189, Process_Duration=2, Process_End_Time=1452791799191, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:39,199 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791798944_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:39,200 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:39,199 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791798944_0001_00_000001
2016-01-14 17:16:39,200 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791798944_0001_1_00_000000_0
2016-01-14 17:16:39,201 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:39,201 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791798944_0001_1][Event:TASK_FINISHED]: vertexName=B0211B94D940448DABF0062F352EBFBF, taskId=task_1452791798944_0001_1_00_000000, startTime=1452791799131, finishTime=1452791799201, timeTaken=70, status=SUCCEEDED, successfulAttemptID=attempt_1452791798944_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=18, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751828992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791799189, Process_Duration=2, Process_End_Time=1452791799191, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:39,201 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791798944_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:39,201 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791798944_0001_1_00 [B0211B94D940448DABF0062F352EBFBF] : 1
2016-01-14 17:16:39,201 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791798944_0001_1_00 [B0211B94D940448DABF0062F352EBFBF]
2016-01-14 17:16:39,202 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:39,202 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791798944_0001_1][Event:VERTEX_FINISHED]: vertexName=B0211B94D940448DABF0062F352EBFBF, vertexId=vertex_1452791798944_0001_1_00, initRequestedTime=1452791799107, initedTime=1452791799119, startRequestedTime=1452791799120, startedTime=1452791799120, finishTime=1452791799201, timeTaken=81, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=18, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751828992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791799189, Process_Duration=2, Process_End_Time=1452791799191, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791799131, firstTasksToStart=[ task_1452791798944_0001_1_00_000000 ], lastTaskFinishTime=1452791799199, lastTasksToFinish=[ task_1452791798944_0001_1_00_000000 ], minTaskDuration=68, maxTaskDuration=68, avgTaskDuration=68.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791798944_0001_1_00_000000 ], longestDurationTasks=[ task_1452791798944_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:39,202 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791798944_0001_1_00 [B0211B94D940448DABF0062F352EBFBF] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:39,202 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791798944_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:39,202 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791798944_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791798944_0001_00_000001, stoppedTime=1452791799202, exitStatus=0
2016-01-14 17:16:39,202 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:39,203 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791798944_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:39,203 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791798944_0001_1_01 [CA227A3CEC4D44AF8E6A8D6AFA5562BB] attempt: attempt_1452791798944_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:39,203 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: CA227A3CEC4D44AF8E6A8D6AFA5562BB
2016-01-14 17:16:39,203 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791798944_0001_1_01 [CA227A3CEC4D44AF8E6A8D6AFA5562BB]
2016-01-14 17:16:39,203 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791798944_0001_1_00 [B0211B94D940448DABF0062F352EBFBF] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:39,203 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791798944_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:39,204 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791798944_0001_1][Event:TASK_STARTED]: vertexName=CA227A3CEC4D44AF8E6A8D6AFA5562BB, taskId=task_1452791798944_0001_1_01_000000, scheduledTime=1452791799203, launchTime=1452791799203
2016-01-14 17:16:39,204 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:39,204 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791798944_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:39,204 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791798944_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:39,204 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791798944_0001_1_01 [CA227A3CEC4D44AF8E6A8D6AFA5562BB]
2016-01-14 17:16:39,204 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [A61B784B8FD24706A1FC8E022F4F10F2/BE726C1C752F420FB26FA3C74B6505F5] every/(1/1) ...platform/0_VALUES_RESULTS, VertexName: CA227A3CEC4D44AF8E6A8D6AFA5562BB, VertexParallelism: 1, TaskAttemptID:attempt_1452791798944_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B0211B94D940448DABF0062F352EBFBF, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=FBC0BCE68E334493BB905AB510537591, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:39,205 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791798944_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:39,205 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:39,205 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791798944_0001_00_000002
2016-01-14 17:16:39,205 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791798944_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:39,206 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791798944_0001_00_000002
2016-01-14 17:16:39,207 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:39,208 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:39,208 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791798944_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791798944_0001_00_000002, launchTime=1452791799208
2016-01-14 17:16:39,208 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791798944_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:39,209 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791798944_0001_00_000002
2016-01-14 17:16:39,210 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791798944_0001_1_01_000000_0] to container: [container_1452791798944_0001_00_000002]
2016-01-14 17:16:39,210 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791798944_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:39,210 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791798944_0001_00_000002 given task: attempt_1452791798944_0001_1_01_000000_0
2016-01-14 17:16:39,210 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791798944_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791798944_0001_1_01_000000_0
2016-01-14 17:16:39,210 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791798944_0001_1_01_000000_0] started. Is using containerId: [container_1452791798944_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:39 Starting to run new task attempt: attempt_1452791798944_0001_1_01_000000_0
2016-01-14 17:16:39,211 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791798944_0001_1_01_000000_0
2016-01-14 17:16:39,211 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:39,211 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791798944_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CA227A3CEC4D44AF8E6A8D6AFA5562BB, taskAttemptId=attempt_1452791798944_0001_1_01_000000_0, startTime=1452791799210, containerId=container_1452791798944_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791798944_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:39,211 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791798944_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:39,211 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791798944_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:39,211 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:39,211 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791798944_0001_00_000001, Exiting
2016-01-14 17:16:39,211 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791798944_0001_00_000001
2016-01-14 17:16:39,211 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:39,211 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:39,212 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:39,212 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:39,212 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:39,212 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [A61B784B8FD24706A1FC8E022F4F10F2/BE726C1C752F420FB26FA3C74B6505F5] every/(1/1) ...platform/0_VALUES_RESULTS, VertexName: CA227A3CEC4D44AF8E6A8D6AFA5562BB, VertexParallelism: 1, TaskAttemptID:attempt_1452791798944_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B0211B94D940448DABF0062F352EBFBF, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=FBC0BCE68E334493BB905AB510537591, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:39,212 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=751828992
2016-01-14 17:16:39,211 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791798944_0001_00_000001
2016-01-14 17:16:39,211 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791798944_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:39,212 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791798944_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:39,212 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791798944_0001_1_01_000000_0
2016-01-14 17:16:39,213 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:39,213 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:39,213 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791798944_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:39,213 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:39,214 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791798944_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:39,214 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791798944_0001_00_000001 completed successfully
2016-01-14 17:16:39,214 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=FBC0BCE68E334493BB905AB510537591, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:39,214 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:39,215 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: FBC0BCE68E334493BB905AB510537591
2016-01-14 17:16:39,215 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:39,215 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: FBC0BCE68E334493BB905AB510537591
2016-01-14 17:16:39,215 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791798944_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:39,216 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B0211B94D940448DABF0062F352EBFBF, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:39,216 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:39,216 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:39,216 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B0211B94D940448DABF0062F352EBFBF
2016-01-14 17:16:39,218 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 676646080, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:39,218 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:39,219 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 676646080, type: INPUT, componentVertexName: B0211B94D940448DABF0062F352EBFBF
2016-01-14 17:16:39,219 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B0211B94D940448DABF0062F352EBFBF
2016-01-14 17:16:39,226 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:39,227 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: FBC0BCE68E334493BB905AB510537591
2016-01-14 17:16:39,226 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CA227A3CEC4D44AF8E6A8D6AFA5562BB, ordinal: 1
2016-01-14 17:16:39,227 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CA227A3CEC4D44AF8E6A8D6AFA5562BB, mem on start (mb), free: 364, total: 717, max: 717
2016-01-14 17:16:39,227 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:39,227 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:39,227 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:39,228 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:39,228 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:39,228 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:39,228 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:39,228 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:39,228 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:39,229 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 676646080, TotalRequestedScaled: 6.245963815384616E8, TotalJVMHeap: 751828992, TotalAvailable: 503725424, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:39,229 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, FBC0BCE68E334493BB905AB510537591, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:39,229 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B0211B94D940448DABF0062F352EBFBF, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=676646080, allocated=503725424
2016-01-14 17:16:39,229 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:39,229 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B0211B94D940448DABF0062F352EBFBF
2016-01-14 17:16:39,230 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:39,230 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:39,230 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B0211B94D940448DABF0062F352EBFBF being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:39,230 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: B0211B94D940448DABF0062F352EBFBF with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:39,231 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=676646080, postMergeMem=0, RuntimeTotalAvailable=503725424. Updated to: ShuffleMem=503725424, postMergeMem: 0
2016-01-14 17:16:39,231 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=503725424, maxSingleShuffleLimit=125931360, mergeThreshold=453352896, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:39,231 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:39,231 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:39,231 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:39,232 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B0211B94D940448DABF0062F352EBFBF
2016-01-14 17:16:39,232 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:39,232 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791798944_0001_1_01_000000_0
2016-01-14 17:16:39,233 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791798944_0001_1_00_000000_0_10183, runDuration: 60837, ]
2016-01-14 17:16:39,234 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791798944_0001_1_00_000000_0_10183] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791799234, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 17:16:39,234 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : B0211B94D940448DABF0062F352EBFBF
2016-01-14 17:16:39,235 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:39,235 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [B0211B94D940448DABF0062F352EBFBF] #31 in 2ms
2016-01-14 17:16:39,235 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [B0211B94D940448DABF0062F352EBFBF] #31, status:false, isInterrupted:false
2016-01-14 17:16:39,235 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:39,236 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:39,236 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:39,236 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:39,236 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:39,236 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : B0211B94D940448DABF0062F352EBFBF
2016-01-14 17:16:39,236 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:39,237 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:39,238 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 3E9D99564E6843F0A58030B96B48A615
2016-01-14 17:16:39,238 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_RESULTS"], id: FBC0BCE68E334493BB905AB510537591
2016-01-14 17:16:39,238 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_RESULTS"]
2016-01-14 17:16:39,238 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 3E9D99564E6843F0A58030B96B48A615, for 1 inputs
2016-01-14 17:16:39,238 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CA227A3CEC4D44AF8E6A8D6AFA5562BB, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:39,239 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@41146e66; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@245c0beb
2016-01-14 17:16:39,240 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:39,240 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(92)) - caught throwable
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:39,240 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:39,241 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:39,241 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791798944_0001_1_01_000000_0
2016-01-14 17:16:39,241 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791798944_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:39,242 INFO  task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1452791798944_0001_1_01_000000_0
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:39,242 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:39,243 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791798944_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CA227A3CEC4D44AF8E6A8D6AFA5562BB, taskAttemptId=attempt_1452791798944_0001_1_01_000000_0, startTime=1452791799210, finishTime=1452791799242, timeTaken=32, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751828992, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791799238, Process_Duration=4, Process_End_Time=1452791799242, Read_Duration=0, Tuples_Read=1
2016-01-14 17:16:39,243 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791798944_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-01-14 17:16:39,243 INFO  impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1452791798944_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-01-14 17:16:39,243 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791798944_0001_1][Event:TASK_FINISHED]: vertexName=CA227A3CEC4D44AF8E6A8D6AFA5562BB, taskId=task_1452791798944_0001_1_01_000000, startTime=1452791799210, finishTime=1452791799243, timeTaken=33, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
], counters=Counters: 0
2016-01-14 17:16:39,243 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791798944_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-01-14 17:16:39,243 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:39,243 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791798944_0001_1_01 [CA227A3CEC4D44AF8E6A8D6AFA5562BB] : 1
2016-01-14 17:16:39,244 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:39,244 INFO  impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1452791798944_0001_1_01 [CA227A3CEC4D44AF8E6A8D6AFA5562BB] because task failed: task_1452791798944_0001_1_01_000000
2016-01-14 17:16:39,244 INFO  impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1452791798944_0001_1_01 [CA227A3CEC4D44AF8E6A8D6AFA5562BB] due to trigger: OWN_TASK_FAILURE
2016-01-14 17:16:39,244 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-01-14 17:16:39,244 INFO  impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1452791798944_0001_1_01 [CA227A3CEC4D44AF8E6A8D6AFA5562BB]
2016-01-14 17:16:39,244 INFO  impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=FBC0BCE68E334493BB905AB510537591, vertexId=vertex_1452791798944_0001_1_01 [CA227A3CEC4D44AF8E6A8D6AFA5562BB]
2016-01-14 17:16:39,245 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:39,245 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791798944_0001_1][Event:VERTEX_FINISHED]: vertexName=CA227A3CEC4D44AF8E6A8D6AFA5562BB, vertexId=vertex_1452791798944_0001_1_01, initRequestedTime=1452791799108, initedTime=1452791799116, startRequestedTime=1452791799122, startedTime=1452791799122, finishTime=1452791799245, timeTaken=123, status=FAILED, diagnostics=Task failed, taskId=task_1452791798944_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1452791798944_0001_1_01 [CA227A3CEC4D44AF8E6A8D6AFA5562BB] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1452791799210, firstTasksToStart=[ task_1452791798944_0001_1_01_000000 ], lastTaskFinishTime=1452791799242, lastTasksToFinish=[ task_1452791798944_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[  ], longestDurationTasks=[  ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-01-14 17:16:39,245 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791798944_0001_1_01 [CA227A3CEC4D44AF8E6A8D6AFA5562BB] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-01-14 17:16:39,246 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791798944_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791798944_0001_00_000002, stoppedTime=1452791799246, exitStatus=0
2016-01-14 17:16:39,246 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:39,246 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791798944_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:39,246 INFO  node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1452791798944_0001_1_01_000000_0 failed: true container: container_1452791798944_0001_00_000002 numFailedTAs: 0
2016-01-14 17:16:39,247 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791798944_0001_1_01 [CA227A3CEC4D44AF8E6A8D6AFA5562BB] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-01-14 17:16:39,247 INFO  impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-01-14 17:16:39,247 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791798944_0001_1
2016-01-14 17:16:39,247 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: FBC0BCE68E334493BB905AB510537591 for vertex: vertex_1452791798944_0001_1_01 [CA227A3CEC4D44AF8E6A8D6AFA5562BB]
2016-01-14 17:16:39,247 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1452791798944_0001_1_00 [B0211B94D940448DABF0062F352EBFBF]
2016-01-14 17:16:39,248 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791798944_0001_1, queueSize=0
2016-01-14 17:16:39,248 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751828992, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791799238, Process_Duration=4, Process_End_Time=1452791799242, Read_Duration=0, Tuples_Read=1]]
2016-01-14 17:16:39,249 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791798944_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791798944_0001_1, startTime=1452791799107, finishTime=1452791799247, timeTaken=140, status=FAILED, diagnostics=Vertex failed, vertexName=CA227A3CEC4D44AF8E6A8D6AFA5562BB, vertexId=vertex_1452791798944_0001_1_01, diagnostics=[Task failed, taskId=task_1452791798944_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1452791798944_0001_1_01 [CA227A3CEC4D44AF8E6A8D6AFA5562BB] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=18, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751828992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791799189, Process_Duration=2, Process_End_Time=1452791799191, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:39,249 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791798944_0001_1 finished with state: FAILED
2016-01-14 17:16:39,249 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791798944_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:39,249 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791798944_0001_00_000002. Relying on regular task shutdown for it to end
2016-01-14 17:16:39 Completed Dag: dag_1452791798944_0001_1
2016-01-14 17:16:39,249 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791798944_0001_1, dagState=FAILED
2016-01-14 17:16:39,249 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791798944_0001_1_post
2016-01-14 17:16:39,249 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:39,250 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:39,250 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791798944_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:39,249 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CA227A3CEC4D44AF8E6A8D6AFA5562BB, mem on close (mb), free: 360, total: 717, max: 717
2016-01-14 17:16:39,250 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=CA227A3CEC4D44AF8E6A8D6AFA5562BB, index=1
2016-01-14 17:16:39,250 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: B0211B94D940448DABF0062F352EBFBF
2016-01-14 17:16:39,248 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:39,252 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:39,252 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:39,252 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:39,252 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:39,253 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:39,253 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=CA227A3CEC4D44AF8E6A8D6AFA5562BB, sourceVertex=B0211B94D940448DABF0062F352EBFBF
2016-01-14 17:16:39,253 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=CA227A3CEC4D44AF8E6A8D6AFA5562BB, sourceVertex=FBC0BCE68E334493BB905AB510537591
2016-01-14 17:16:39,253 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791798944_0001_00_000002
2016-01-14 17:16:39,253 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791798944_0001_00_000002
2016-01-14 17:16:39,253 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791798944_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-01-14 17:16:39,253 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1452791798944_0001_00_000002 completed but with errors
2016-01-14 17:16:39,254 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1452791798944_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-01-14 17:16:39,254 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791798944_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:39,254 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791798944_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-01-14 17:16:39,255 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...platform/0_VALUES_RESULTS, applicationId=application_1452791798944_0001
2016-01-14 17:16:39,256 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:39,256 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:39,256 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:39,256 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:39,256 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:39,256 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:39,257 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:39,257 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:39,257 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:39,257 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:39,257 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:39,257 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:39,258 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:39,262 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A61B784B8FD24706A1FC8E022F4F10F2/BE726C1C752F420FB26FA3C74B6505F5/.tez/application_1452791798944_0001
2016-01-14 17:16:39,262 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping all jobs
2016-01-14 17:16:39,262 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping: (1/1) ...platform/0_VALUES_RESULTS
2016-01-14 17:16:39,263 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopped all jobs
2016-01-14 17:16:39,263 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.337
2016-01-14 17:16:39,263 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_VALUES_RESULTS
2016-01-14 17:16:39,284 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:39,284 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:39,303 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 17:16:39,303 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 17:16:39,320 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-01-14 17:16:39,329 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.019
2016-01-14 17:16:39,329 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:39,329 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:39,329 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:39,342 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:39,347 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_ALL
2016-01-14 17:16:39,388 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:39,389 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:39,389 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_ALL"]
2016-01-14 17:16:39,389 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:39,390 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:39,391 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:39,391 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...mblyplatform/0_VALUES_ALL
2016-01-14 17:16:39,391 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:39,402 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:39,407 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:39,407 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:39,407 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:39,416 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A88383B8741345B78841B3EFE275B1CF/A116BE01D5DE437DA6B821BCC432635E/.tez/application_1452791799407_0001 doesn't exist and is created
2016-01-14 17:16:39,430 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:39,431 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:39,431 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A88383B8741345B78841B3EFE275B1CF/A116BE01D5DE437DA6B821BCC432635E/.tez/application_1452791799407_0001
2016-01-14 17:16:39,434 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791799407_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:39,444 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:39,445 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:39,448 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:39,448 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:39,449 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A88383B8741345B78841B3EFE275B1CF/A116BE01D5DE437DA6B821BCC432635E/.tez/application_1452791799407_0001/localmode-log-dir
2016-01-14 17:16:39,449 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A88383B8741345B78841B3EFE275B1CF/A116BE01D5DE437DA6B821BCC432635E/.tez/application_1452791799407_0001/localmode-log-dir/history.txt.appattempt_1452791799407_0001_000000, maxErrors=10
2016-01-14 17:16:39,449 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:39,449 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791799407_0001_000000, appSubmitTime=1452791799434, launchTime=1452791799434
2016-01-14 17:16:39,451 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:39,452 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:39,454 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:39,454 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:39,460 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:60562
2016-01-14 17:16:39,462 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:39,463 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791799407_0001_000000, startTime=1452791799463
2016-01-14 17:16:39,463 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:39,463 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:39,531 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:39,531 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:39,531 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...mblyplatform/0_VALUES_ALL, applicationId=application_1452791799407_0001, dagName=[A88383B8741345B78841B3EFE275B1CF/A116BE01D5DE437DA6B821BCC432635E] every/(1/1) ...mblyplatform/0_VALUES_ALL
2016-01-14 17:16:39,546 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [A88383B8741345B78841B3EFE275B1CF/A116BE01D5DE437DA6B821BCC432635E] every/(1/1) ...mblyplatform/0_VALUES_ALL
2016-01-14 17:16:39,548 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791799407_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A88383B8741345B78841B3EFE275B1CF/A116BE01D5DE437DA6B821BCC432635E/.tez/application_1452791799407_0001/localmode-log-dir/dag_1452791799407_0001_1.dot
2016-01-14 17:16:39,548 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A88383B8741345B78841B3EFE275B1CF/A116BE01D5DE437DA6B821BCC432635E/.tez/application_1452791799407_0001/localmode-log-dir/dag_1452791799407_0001_1-tez-dag.pb.txt
2016-01-14 17:16:39,554 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791799407_0001_1
2016-01-14 17:16:39,554 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:39,554 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [A88383B8741345B78841B3EFE275B1CF/A116BE01D5DE437DA6B821BCC432635E] every/(1/1) ...mblyplatform/0_VALUES_ALL
2016-01-14 17:16:39 Running Dag: dag_1452791799407_0001_1
2016-01-14 17:16:39,568 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791799407_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791799407_0001_1, submitTime=1452791799546
2016-01-14 17:16:39,568 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:39,569 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex CE0BAE97783048E382CC297DC737202C
2016-01-14 17:16:39,570 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:39,571 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791799407_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791799407_0001_1, initTime=1452791799569
2016-01-14 17:16:39,571 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791799407_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:39,571 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:39,571 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791799407_0001_1][Event:DAG_STARTED]: dagID=dag_1452791799407_0001_1, startTime=1452791799571
2016-01-14 17:16:39,571 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:39,571 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791799407_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:39,571 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:39,571 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: BDEF7402445C4710BC6A0365EEF68CB3 : {23CB0A3172AD44C4A6EFECEB90CBF0BC={InputName=23CB0A3172AD44C4A6EFECEB90CBF0BC}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:39,572 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 23CB0A3172AD44C4A6EFECEB90CBF0BC, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:39,572 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791799407_0001_1_00 [BDEF7402445C4710BC6A0365EEF68CB3]
2016-01-14 17:16:39,572 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791799407_0001_1_00 [BDEF7402445C4710BC6A0365EEF68CB3]
2016-01-14 17:16:39,572 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791799407_0001_1_00 [BDEF7402445C4710BC6A0365EEF68CB3]
2016-01-14 17:16:39,572 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791799407_0001_1_00 [BDEF7402445C4710BC6A0365EEF68CB3]. Starting root input initializers: 1
2016-01-14 17:16:39,572 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791799407_0001_1_00 [BDEF7402445C4710BC6A0365EEF68CB3] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:39,572 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791799407_0001_1_01 [CE0BAE97783048E382CC297DC737202C]
2016-01-14 17:16:39,574 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 23CB0A3172AD44C4A6EFECEB90CBF0BC on vertex vertex_1452791799407_0001_1_00 [BDEF7402445C4710BC6A0365EEF68CB3]
2016-01-14 17:16:39,574 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...mblyplatform/0_VALUES_ALL, applicationId=application_1452791799407_0001, dagName=[A88383B8741345B78841B3EFE275B1CF/A116BE01D5DE437DA6B821BCC432635E] every/(1/1) ...mblyplatform/0_VALUES_ALL
2016-01-14 17:16:39,575 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:39,575 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791799407_0001_1_01 [CE0BAE97783048E382CC297DC737202C]
2016-01-14 17:16:39,575 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:39,576 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 23CB0A3172AD44C4A6EFECEB90CBF0BC asking for 1 tasks. Headroom: 717 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:39,576 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:39,576 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791799407_0001_1_01 [CE0BAE97783048E382CC297DC737202C]
2016-01-14 17:16:39,576 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791799407_0001_1_01 [CE0BAE97783048E382CC297DC737202C]
2016-01-14 17:16:39,577 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=B888A9F34EEC42DF8486874FF344BF5F, vertexId=vertex_1452791799407_0001_1_01 [CE0BAE97783048E382CC297DC737202C], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:39,577 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=B888A9F34EEC42DF8486874FF344BF5F, vertexId=vertex_1452791799407_0001_1_01 [CE0BAE97783048E382CC297DC737202C]
2016-01-14 17:16:39,577 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:39,578 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:39,578 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=B888A9F34EEC42DF8486874FF344BF5F, vertexName=CE0BAE97783048E382CC297DC737202C, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:39,579 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=B888A9F34EEC42DF8486874FF344BF5F, vertexId=vertex_1452791799407_0001_1_01 [CE0BAE97783048E382CC297DC737202C]
2016-01-14 17:16:39,579 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 23CB0A3172AD44C4A6EFECEB90CBF0BC on vertex vertex_1452791799407_0001_1_00 [BDEF7402445C4710BC6A0365EEF68CB3]
2016-01-14 17:16:39,580 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:39,580 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791799407_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CE0BAE97783048E382CC297DC737202C, vertexId=vertex_1452791799407_0001_1_01, initRequestedTime=1452791799572, initedTime=1452791799580, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:39,580 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791799407_0001_1_01 [CE0BAE97783048E382CC297DC737202C] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:39,580 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791799407_0001_1_00 [BDEF7402445C4710BC6A0365EEF68CB3] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:39,580 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {23CB0A3172AD44C4A6EFECEB90CBF0BC=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:39,580 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791799407_0001_1_00 [BDEF7402445C4710BC6A0365EEF68CB3] parallelism set to 1
2016-01-14 17:16:39,581 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791799407_0001_1_00 [BDEF7402445C4710BC6A0365EEF68CB3] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:39,581 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791799407_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791799407_0001_1_00, eventCount=1
2016-01-14 17:16:39,582 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:39,582 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791799407_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BDEF7402445C4710BC6A0365EEF68CB3, vertexId=vertex_1452791799407_0001_1_00, initRequestedTime=1452791799571, initedTime=1452791799581, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:39,582 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791799407_0001_1_00 [BDEF7402445C4710BC6A0365EEF68CB3] with distanceFromRoot: 0
2016-01-14 17:16:39,582 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791799407_0001_1_00 [BDEF7402445C4710BC6A0365EEF68CB3] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:39,582 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in BDEF7402445C4710BC6A0365EEF68CB3
2016-01-14 17:16:39,582 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791799407_0001_1_00 [BDEF7402445C4710BC6A0365EEF68CB3]
2016-01-14 17:16:39,583 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:39,583 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791799407_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791799407_0001_1_00, startRequestedTime=1452791799582, startedTime=1452791799582
2016-01-14 17:16:39,583 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791799407_0001_1_00 [BDEF7402445C4710BC6A0365EEF68CB3] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:39,584 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:39,584 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791799407_0001_1][Event:TASK_STARTED]: vertexName=BDEF7402445C4710BC6A0365EEF68CB3, taskId=task_1452791799407_0001_1_00_000000, scheduledTime=1452791799583, launchTime=1452791799583
2016-01-14 17:16:39,584 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791799407_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:39,584 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791799407_0001_1_00 for vertex: vertex_1452791799407_0001_1_01 [CE0BAE97783048E382CC297DC737202C] numStartedSources: 1 numSources: 1
2016-01-14 17:16:39,584 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791799407_0001_1_01 [CE0BAE97783048E382CC297DC737202C] with distanceFromRoot: 1
2016-01-14 17:16:39,585 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791799407_0001_1_00 [BDEF7402445C4710BC6A0365EEF68CB3]
2016-01-14 17:16:39,585 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: BDEF7402445C4710BC6A0365EEF68CB3 in vertex: CE0BAE97783048E382CC297DC737202C
2016-01-14 17:16:39,585 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: CE0BAE97783048E382CC297DC737202C with 1 source tasks and 1 pending tasks
2016-01-14 17:16:39,585 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:39,585 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791799407_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791799407_0001_1_01, startRequestedTime=1452791799585, startedTime=1452791799585
2016-01-14 17:16:39,585 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791799407_0001_1_01 [CE0BAE97783048E382CC297DC737202C] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:39,586 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [A88383B8741345B78841B3EFE275B1CF/A116BE01D5DE437DA6B821BCC432635E] every/(1/1) ...mblyplatform/0_VALUES_ALL, VertexName: BDEF7402445C4710BC6A0365EEF68CB3, VertexParallelism: 1, TaskAttemptID:attempt_1452791799407_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=23CB0A3172AD44C4A6EFECEB90CBF0BC, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CE0BAE97783048E382CC297DC737202C, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:39,586 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791799407_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:39,586 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:39,587 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:39,587 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:39,587 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:39,588 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791799407_0001_00_000001
2016-01-14 17:16:39,588 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791799407_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:39,589 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791799407_0001_00_000001
2016-01-14 17:16:39,589 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:39,589 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791799407_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:39,590 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:39,590 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791799407_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791799407_0001_00_000001, launchTime=1452791799590
2016-01-14 17:16:39,590 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791799407_0001_00_000001
2016-01-14 17:16:39,591 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791799407_0001_1_00_000000_0] to container: [container_1452791799407_0001_00_000001]
2016-01-14 17:16:39,591 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791799407_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:39,591 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791799407_0001_00_000001 given task: attempt_1452791799407_0001_1_00_000000_0
2016-01-14 17:16:39,591 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791799407_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791799407_0001_1_00_000000_0
2016-01-14 17:16:39,591 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791799407_0001_1_00_000000_0] started. Is using containerId: [container_1452791799407_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:39,591 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791799407_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BDEF7402445C4710BC6A0365EEF68CB3, taskAttemptId=attempt_1452791799407_0001_1_00_000000_0, startTime=1452791799591, containerId=container_1452791799407_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791799407_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:39,591 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791799407_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:39 Starting to run new task attempt: attempt_1452791799407_0001_1_00_000000_0
2016-01-14 17:16:39,592 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791799407_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:39,592 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791799407_0001_1_00_000000_0
2016-01-14 17:16:39,592 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:39,592 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:39,592 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:39,592 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:39,592 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:39,593 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:39,593 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:39,593 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [A88383B8741345B78841B3EFE275B1CF/A116BE01D5DE437DA6B821BCC432635E] every/(1/1) ...mblyplatform/0_VALUES_ALL, VertexName: BDEF7402445C4710BC6A0365EEF68CB3, VertexParallelism: 1, TaskAttemptID:attempt_1452791799407_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=23CB0A3172AD44C4A6EFECEB90CBF0BC, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CE0BAE97783048E382CC297DC737202C, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:39,593 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=751828992
2016-01-14 17:16:39,593 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791799407_0001_1_00_000000_0
2016-01-14 17:16:39,594 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:39,594 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:39,594 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791799407_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:39,595 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:39,597 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:39,594 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=23CB0A3172AD44C4A6EFECEB90CBF0BC, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:39,598 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:39,598 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 23CB0A3172AD44C4A6EFECEB90CBF0BC
2016-01-14 17:16:39,599 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 23CB0A3172AD44C4A6EFECEB90CBF0BC
2016-01-14 17:16:39,599 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=CE0BAE97783048E382CC297DC737202C, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:39,599 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:39,600 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: CE0BAE97783048E382CC297DC737202C
2016-01-14 17:16:39,601 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791799407_0001, with dag id: dag_1452791799407_0001_1
2016-01-14 17:16:39,602 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:39,602 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 23CB0A3172AD44C4A6EFECEB90CBF0BC
2016-01-14 17:16:39,602 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 23CB0A3172AD44C4A6EFECEB90CBF0BC
2016-01-14 17:16:39,602 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BDEF7402445C4710BC6A0365EEF68CB3, ordinal: 0
2016-01-14 17:16:39,603 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BDEF7402445C4710BC6A0365EEF68CB3, mem on start (mb), free: 288, total: 717, max: 717
2016-01-14 17:16:39,603 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:39,603 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:39,603 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:39,603 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:39,603 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: CE0BAE97783048E382CC297DC737202C
2016-01-14 17:16:39,604 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: CE0BAE97783048E382CC297DC737202C
2016-01-14 17:16:39,604 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:39,604 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:39,604 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:39,605 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:39,605 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:39,605 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:39,605 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 751828992, TotalAvailable: 503725424, TotalRequested/TotalJVMHeap:0.14
2016-01-14 17:16:39,605 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 23CB0A3172AD44C4A6EFECEB90CBF0BC, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:39,605 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, CE0BAE97783048E382CC297DC737202C, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:39,605 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:39,605 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 23CB0A3172AD44C4A6EFECEB90CBF0BC being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:39,605 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:39,605 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:39,605 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 23CB0A3172AD44C4A6EFECEB90CBF0BC
2016-01-14 17:16:39,606 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:39,606 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 23CB0A3172AD44C4A6EFECEB90CBF0BC
2016-01-14 17:16:39,606 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:39,607 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791799407_0001_1_00_000000_0
2016-01-14 17:16:39,607 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:39,608 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@55748a55
2016-01-14 17:16:39,609 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@55748a55
2016-01-14 17:16:39,610 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:39,610 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:39,610 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:39,611 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 23CB0A3172AD44C4A6EFECEB90CBF0BC
2016-01-14 17:16:39,611 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: C253ADF8F2664624B3723393C89B19E8
2016-01-14 17:16:39,611 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] C253ADF8F2664624B3723393C89B19E8
2016-01-14 17:16:39,612 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@59a176fc; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@778ac85c; comparator=cascading.tuple.hadoop.util.TupleComparator@38684d54; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:39,612 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:39,612 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:39,612 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:39,713 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:39,713 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:39,713 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:39,713 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:39,714 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:39,714 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:39,714 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:39,714 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BDEF7402445C4710BC6A0365EEF68CB3, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:39,717 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791799407_0001_1_00_000000_0
2016-01-14 17:16:39,717 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BDEF7402445C4710BC6A0365EEF68CB3, mem on close (mb), free: 185, total: 717, max: 717
2016-01-14 17:16:39,717 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:39,717 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:39,717 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:39,717 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:39,720 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:39 Completed running task attempt: attempt_1452791799407_0001_1_00_000000_0
2016-01-14 17:16:39,723 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791799407_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:39,724 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751828992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791799714, Process_Duration=3, Process_End_Time=1452791799717, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:39,724 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791799407_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791799407_0001_1_00, eventCount=1
2016-01-14 17:16:39,724 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:39,724 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:39,724 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:39,724 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791799407_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BDEF7402445C4710BC6A0365EEF68CB3, taskAttemptId=attempt_1452791799407_0001_1_00_000000_0, startTime=1452791799591, finishTime=1452791799724, timeTaken=133, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751828992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791799714, Process_Duration=3, Process_End_Time=1452791799717, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:39,724 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791799407_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:39,725 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791799407_0001_1_00_000000_0
2016-01-14 17:16:39,724 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791799407_0001_00_000001
2016-01-14 17:16:39,725 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:39,725 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791799407_0001_1][Event:TASK_FINISHED]: vertexName=BDEF7402445C4710BC6A0365EEF68CB3, taskId=task_1452791799407_0001_1_00_000000, startTime=1452791799591, finishTime=1452791799725, timeTaken=134, status=SUCCEEDED, successfulAttemptID=attempt_1452791799407_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751828992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791799714, Process_Duration=3, Process_End_Time=1452791799717, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:39,725 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:39,725 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791799407_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:39,727 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791799407_0001_1_00 [BDEF7402445C4710BC6A0365EEF68CB3] : 1
2016-01-14 17:16:39,727 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791799407_0001_1_00 [BDEF7402445C4710BC6A0365EEF68CB3]
2016-01-14 17:16:39,727 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:39,727 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791799407_0001_1][Event:VERTEX_FINISHED]: vertexName=BDEF7402445C4710BC6A0365EEF68CB3, vertexId=vertex_1452791799407_0001_1_00, initRequestedTime=1452791799571, initedTime=1452791799581, startRequestedTime=1452791799582, startedTime=1452791799582, finishTime=1452791799727, timeTaken=145, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751828992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791799714, Process_Duration=3, Process_End_Time=1452791799717, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791799591, firstTasksToStart=[ task_1452791799407_0001_1_00_000000 ], lastTaskFinishTime=1452791799724, lastTasksToFinish=[ task_1452791799407_0001_1_00_000000 ], minTaskDuration=133, maxTaskDuration=133, avgTaskDuration=133.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791799407_0001_1_00_000000 ], longestDurationTasks=[ task_1452791799407_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:39,728 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791799407_0001_1_00 [BDEF7402445C4710BC6A0365EEF68CB3] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:39,728 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791799407_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:39,728 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791799407_0001_1_01 [CE0BAE97783048E382CC297DC737202C] attempt: attempt_1452791799407_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:39,728 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: CE0BAE97783048E382CC297DC737202C
2016-01-14 17:16:39,728 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791799407_0001_1_01 [CE0BAE97783048E382CC297DC737202C]
2016-01-14 17:16:39,728 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791799407_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791799407_0001_00_000001, stoppedTime=1452791799728, exitStatus=0
2016-01-14 17:16:39,728 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:39,728 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791799407_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:39,729 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791799407_0001_1_00 [BDEF7402445C4710BC6A0365EEF68CB3] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:39,729 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791799407_0001_1][Event:TASK_STARTED]: vertexName=CE0BAE97783048E382CC297DC737202C, taskId=task_1452791799407_0001_1_01_000000, scheduledTime=1452791799729, launchTime=1452791799729
2016-01-14 17:16:39,729 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:39,729 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791799407_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:39,729 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791799407_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:39,729 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791799407_0001_1_01 [CE0BAE97783048E382CC297DC737202C]
2016-01-14 17:16:39,729 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791799407_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:39,730 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [A88383B8741345B78841B3EFE275B1CF/A116BE01D5DE437DA6B821BCC432635E] every/(1/1) ...mblyplatform/0_VALUES_ALL, VertexName: CE0BAE97783048E382CC297DC737202C, VertexParallelism: 1, TaskAttemptID:attempt_1452791799407_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BDEF7402445C4710BC6A0365EEF68CB3, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=B888A9F34EEC42DF8486874FF344BF5F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:39,730 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791799407_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:39,730 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:39,730 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791799407_0001_00_000002
2016-01-14 17:16:39,731 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791799407_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:39,732 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791799407_0001_00_000002
2016-01-14 17:16:39,732 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:39,733 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791799407_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:39,733 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791799407_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791799407_0001_00_000002, launchTime=1452791799733
2016-01-14 17:16:39,733 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:39,733 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791799407_0001_00_000002
2016-01-14 17:16:39,734 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791799407_0001_1_01_000000_0] to container: [container_1452791799407_0001_00_000002]
2016-01-14 17:16:39,734 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791799407_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:39,734 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791799407_0001_1_01_000000_0] started. Is using containerId: [container_1452791799407_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:39,734 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791799407_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CE0BAE97783048E382CC297DC737202C, taskAttemptId=attempt_1452791799407_0001_1_01_000000_0, startTime=1452791799734, containerId=container_1452791799407_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791799407_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:39,735 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:39,735 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791799407_0001_00_000002 given task: attempt_1452791799407_0001_1_01_000000_0
2016-01-14 17:16:39,735 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791799407_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:39,735 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791799407_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:39,735 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791799407_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791799407_0001_1_01_000000_0
2016-01-14 17:16:39 Starting to run new task attempt: attempt_1452791799407_0001_1_01_000000_0
2016-01-14 17:16:39,735 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791799407_0001_1_01_000000_0
2016-01-14 17:16:39,736 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:39,736 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:39,736 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:39,736 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:39,736 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:39,736 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:39,737 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [A88383B8741345B78841B3EFE275B1CF/A116BE01D5DE437DA6B821BCC432635E] every/(1/1) ...mblyplatform/0_VALUES_ALL, VertexName: CE0BAE97783048E382CC297DC737202C, VertexParallelism: 1, TaskAttemptID:attempt_1452791799407_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BDEF7402445C4710BC6A0365EEF68CB3, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=B888A9F34EEC42DF8486874FF344BF5F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:39,737 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=751828992
2016-01-14 17:16:39,737 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791799407_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:39,737 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791799407_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:39,737 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791799407_0001_00_000001, Exiting
2016-01-14 17:16:39,738 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791799407_0001_00_000001
2016-01-14 17:16:39,738 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791799407_0001_00_000001
2016-01-14 17:16:39,738 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791799407_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:39,738 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791799407_0001_1_01_000000_0
2016-01-14 17:16:39,739 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:39,739 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:39,739 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791799407_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:39,740 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791799407_0001_00_000001 completed successfully
2016-01-14 17:16:39,740 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:39,740 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791799407_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:39,740 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=BDEF7402445C4710BC6A0365EEF68CB3, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:39,741 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:39,741 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: BDEF7402445C4710BC6A0365EEF68CB3
2016-01-14 17:16:39,742 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 676646080, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:39,742 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:39,743 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 676646080, type: INPUT, componentVertexName: BDEF7402445C4710BC6A0365EEF68CB3
2016-01-14 17:16:39,743 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: BDEF7402445C4710BC6A0365EEF68CB3
2016-01-14 17:16:39,743 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:39,743 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=B888A9F34EEC42DF8486874FF344BF5F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:39,745 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:39,745 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: B888A9F34EEC42DF8486874FF344BF5F
2016-01-14 17:16:39,745 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:39,745 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: B888A9F34EEC42DF8486874FF344BF5F
2016-01-14 17:16:39,749 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CE0BAE97783048E382CC297DC737202C, ordinal: 1
2016-01-14 17:16:39,749 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CE0BAE97783048E382CC297DC737202C, mem on start (mb), free: 180, total: 717, max: 717
2016-01-14 17:16:39,749 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:39,749 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:39,749 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:39,753 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:39,753 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: B888A9F34EEC42DF8486874FF344BF5F
2016-01-14 17:16:39,754 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:39,754 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:39,754 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:39,754 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:39,754 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:39,754 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:39,754 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 676646080, TotalRequestedScaled: 6.245963815384616E8, TotalJVMHeap: 751828992, TotalAvailable: 503725424, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:39,754 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, BDEF7402445C4710BC6A0365EEF68CB3, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=676646080, allocated=503725424
2016-01-14 17:16:39,754 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, B888A9F34EEC42DF8486874FF344BF5F, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:39,754 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:39,755 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: BDEF7402445C4710BC6A0365EEF68CB3 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:39,755 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: BDEF7402445C4710BC6A0365EEF68CB3
2016-01-14 17:16:39,755 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:39,755 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:39,756 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:39,756 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:39,756 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: BDEF7402445C4710BC6A0365EEF68CB3 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:39,756 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=676646080, postMergeMem=0, RuntimeTotalAvailable=503725424. Updated to: ShuffleMem=503725424, postMergeMem: 0
2016-01-14 17:16:39,756 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=503725424, maxSingleShuffleLimit=125931360, mergeThreshold=453352896, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:39,757 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:39,757 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: BDEF7402445C4710BC6A0365EEF68CB3
2016-01-14 17:16:39,758 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:39,759 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791799407_0001_1_01_000000_0
2016-01-14 17:16:39,760 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791799407_0001_1_00_000000_0_10189, runDuration: 122441, ]
2016-01-14 17:16:39,761 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791799407_0001_1_00_000000_0_10189] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791799761, TimeTaken=0, Rate=0.00 MB/s
2016-01-14 17:16:39,761 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : BDEF7402445C4710BC6A0365EEF68CB3
2016-01-14 17:16:39,762 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:39,762 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [BDEF7402445C4710BC6A0365EEF68CB3] #32 in 1ms
2016-01-14 17:16:39,762 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:39,762 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: C253ADF8F2664624B3723393C89B19E8
2016-01-14 17:16:39,763 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_ALL"], id: B888A9F34EEC42DF8486874FF344BF5F
2016-01-14 17:16:39,762 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [BDEF7402445C4710BC6A0365EEF68CB3] #32, status:false, isInterrupted:false
2016-01-14 17:16:39,763 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_ALL"]
2016-01-14 17:16:39,763 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] C253ADF8F2664624B3723393C89B19E8, for 1 inputs
2016-01-14 17:16:39,763 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:39,764 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:39,764 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:39,764 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:39,764 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:39,764 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : BDEF7402445C4710BC6A0365EEF68CB3
2016-01-14 17:16:39,764 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:39,764 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CE0BAE97783048E382CC297DC737202C, all 1 inputs ready in: 00:00:00.001
2016-01-14 17:16:39,765 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@56272054; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@22bb96fc
2016-01-14 17:16:39,766 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:39,766 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(92)) - caught throwable
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:39,766 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:39,766 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:39,767 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791799407_0001_1_01_000000_0
2016-01-14 17:16:39,767 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791799407_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:39,767 INFO  task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1452791799407_0001_1_01_000000_0
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:39,770 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751828992, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791799764, Process_Duration=3, Process_End_Time=1452791799767, Read_Duration=1, Tuples_Read=1]]
2016-01-14 17:16:39,770 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:39,770 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791799407_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CE0BAE97783048E382CC297DC737202C, taskAttemptId=attempt_1452791799407_0001_1_01_000000_0, startTime=1452791799734, finishTime=1452791799770, timeTaken=36, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751828992, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791799764, Process_Duration=3, Process_End_Time=1452791799767, Read_Duration=1, Tuples_Read=1
2016-01-14 17:16:39,770 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791799407_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-01-14 17:16:39,770 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CE0BAE97783048E382CC297DC737202C, mem on close (mb), free: 177, total: 717, max: 717
2016-01-14 17:16:39,770 INFO  impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1452791799407_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-01-14 17:16:39,770 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=CE0BAE97783048E382CC297DC737202C, index=1
2016-01-14 17:16:39,771 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:39,771 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: BDEF7402445C4710BC6A0365EEF68CB3
2016-01-14 17:16:39,771 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791799407_0001_1][Event:TASK_FINISHED]: vertexName=CE0BAE97783048E382CC297DC737202C, taskId=task_1452791799407_0001_1_01_000000, startTime=1452791799734, finishTime=1452791799771, timeTaken=37, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
], counters=Counters: 0
2016-01-14 17:16:39,771 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791799407_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-01-14 17:16:39,771 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791799407_0001_1_01 [CE0BAE97783048E382CC297DC737202C] : 1
2016-01-14 17:16:39,771 INFO  impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1452791799407_0001_1_01 [CE0BAE97783048E382CC297DC737202C] because task failed: task_1452791799407_0001_1_01_000000
2016-01-14 17:16:39,772 INFO  impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1452791799407_0001_1_01 [CE0BAE97783048E382CC297DC737202C] due to trigger: OWN_TASK_FAILURE
2016-01-14 17:16:39,771 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:39,772 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-01-14 17:16:39,772 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:39,772 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:39,772 INFO  impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1452791799407_0001_1_01 [CE0BAE97783048E382CC297DC737202C]
2016-01-14 17:16:39,772 INFO  impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=B888A9F34EEC42DF8486874FF344BF5F, vertexId=vertex_1452791799407_0001_1_01 [CE0BAE97783048E382CC297DC737202C]
2016-01-14 17:16:39,774 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791799407_0001_1][Event:VERTEX_FINISHED]: vertexName=CE0BAE97783048E382CC297DC737202C, vertexId=vertex_1452791799407_0001_1_01, initRequestedTime=1452791799572, initedTime=1452791799580, startRequestedTime=1452791799585, startedTime=1452791799585, finishTime=1452791799773, timeTaken=188, status=FAILED, diagnostics=Task failed, taskId=task_1452791799407_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1452791799407_0001_1_01 [CE0BAE97783048E382CC297DC737202C] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1452791799734, firstTasksToStart=[ task_1452791799407_0001_1_01_000000 ], lastTaskFinishTime=1452791799770, lastTasksToFinish=[ task_1452791799407_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[  ], longestDurationTasks=[  ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-01-14 17:16:39,774 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791799407_0001_1_01 [CE0BAE97783048E382CC297DC737202C] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-01-14 17:16:39,774 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791799407_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791799407_0001_00_000002, stoppedTime=1452791799774, exitStatus=0
2016-01-14 17:16:39,774 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791799407_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:39,774 INFO  node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1452791799407_0001_1_01_000000_0 failed: true container: container_1452791799407_0001_00_000002 numFailedTAs: 0
2016-01-14 17:16:39,774 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791799407_0001_1_01 [CE0BAE97783048E382CC297DC737202C] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-01-14 17:16:39,774 INFO  impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-01-14 17:16:39,774 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791799407_0001_1
2016-01-14 17:16:39,774 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1452791799407_0001_1_00 [BDEF7402445C4710BC6A0365EEF68CB3]
2016-01-14 17:16:39,775 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: B888A9F34EEC42DF8486874FF344BF5F for vertex: vertex_1452791799407_0001_1_01 [CE0BAE97783048E382CC297DC737202C]
2016-01-14 17:16:39,775 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:39,776 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:39,776 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791799407_0001_1, queueSize=0
2016-01-14 17:16:39,776 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791799407_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791799407_0001_1, startTime=1452791799571, finishTime=1452791799775, timeTaken=204, status=FAILED, diagnostics=Vertex failed, vertexName=CE0BAE97783048E382CC297DC737202C, vertexId=vertex_1452791799407_0001_1_01, diagnostics=[Task failed, taskId=task_1452791799407_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1452791799407_0001_1_01 [CE0BAE97783048E382CC297DC737202C] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751828992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791799714, Process_Duration=3, Process_End_Time=1452791799717, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:39,776 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:39,777 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791799407_0001_1 finished with state: FAILED
2016-01-14 17:16:39,777 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791799407_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:39 Completed Dag: dag_1452791799407_0001_1
2016-01-14 17:16:39,778 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791799407_0001_1, dagState=FAILED
2016-01-14 17:16:39,778 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791799407_0001_1_post
2016-01-14 17:16:39,778 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:39,778 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:39,778 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:39,778 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:39,779 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:39,779 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=CE0BAE97783048E382CC297DC737202C, sourceVertex=BDEF7402445C4710BC6A0365EEF68CB3
2016-01-14 17:16:39,779 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=CE0BAE97783048E382CC297DC737202C, sourceVertex=B888A9F34EEC42DF8486874FF344BF5F
2016-01-14 17:16:39,779 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791799407_0001_00_000002
2016-01-14 17:16:39,779 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791799407_0001_00_000002
2016-01-14 17:16:39,780 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791799407_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-01-14 17:16:39,780 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1452791799407_0001_00_000002 completed but with errors
2016-01-14 17:16:39,780 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1452791799407_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-01-14 17:16:39,780 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791799407_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:16:39,780 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791799407_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-01-14 17:16:39,780 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452791799407_0001_00_000002
2016-01-14 17:16:39,790 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...mblyplatform/0_VALUES_ALL, applicationId=application_1452791799407_0001
2016-01-14 17:16:39,791 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:39,792 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:39,792 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:39,792 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:39,792 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:39,809 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:39,810 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:39,810 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:39,810 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:39,810 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:39,810 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:39,810 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:39,810 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:39,815 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A88383B8741345B78841B3EFE275B1CF/A116BE01D5DE437DA6B821BCC432635E/.tez/application_1452791799407_0001
2016-01-14 17:16:39,815 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping all jobs
2016-01-14 17:16:39,816 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping: (1/1) ...mblyplatform/0_VALUES_ALL
2016-01-14 17:16:39,816 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopped all jobs
2016-01-14 17:16:39,816 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.428
2016-01-14 17:16:39,816 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_VALUES_ALL
2016-01-14 17:16:39,842 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:39,842 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:39,862 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 17:16:39,862 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 17:16:39,873 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.009
2016-01-14 17:16:39,881 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.017
2016-01-14 17:16:39,882 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:39,882 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:39,882 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:39,896 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:39,901 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_REPLACE
2016-01-14 17:16:39,943 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:39,944 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:39,944 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_REPLACE"]
2016-01-14 17:16:39,944 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:39,944 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:39,945 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:39,945 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...platform/0_VALUES_REPLACE
2016-01-14 17:16:39,945 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:39,956 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:39,961 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:39,962 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:39,962 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:39,971 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/D0E4EE589CFB4119A517E1FC43A796C1/C7F31E80A5AD43E3977ECDACF039AA04/.tez/application_1452791799961_0001 doesn't exist and is created
2016-01-14 17:16:39,990 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:39,990 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:39,990 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/D0E4EE589CFB4119A517E1FC43A796C1/C7F31E80A5AD43E3977ECDACF039AA04/.tez/application_1452791799961_0001
2016-01-14 17:16:39,993 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791799961_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:40,004 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:40,004 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:40,006 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:40,006 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:40,006 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/D0E4EE589CFB4119A517E1FC43A796C1/C7F31E80A5AD43E3977ECDACF039AA04/.tez/application_1452791799961_0001/localmode-log-dir
2016-01-14 17:16:40,006 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/D0E4EE589CFB4119A517E1FC43A796C1/C7F31E80A5AD43E3977ECDACF039AA04/.tez/application_1452791799961_0001/localmode-log-dir/history.txt.appattempt_1452791799961_0001_000000, maxErrors=10
2016-01-14 17:16:40,006 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:40,007 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791799961_0001_000000, appSubmitTime=1452791799993, launchTime=1452791799993
2016-01-14 17:16:40,010 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:40,012 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:40,013 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:40,013 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:40,017 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:40226
2016-01-14 17:16:40,018 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:40,019 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791799961_0001_000000, startTime=1452791800019
2016-01-14 17:16:40,019 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:40,019 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:40,090 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:40,090 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:40,090 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...platform/0_VALUES_REPLACE, applicationId=application_1452791799961_0001, dagName=[D0E4EE589CFB4119A517E1FC43A796C1/C7F31E80A5AD43E3977ECDACF039AA04] every/(1/1) ...platform/0_VALUES_REPLACE
2016-01-14 17:16:40,105 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [D0E4EE589CFB4119A517E1FC43A796C1/C7F31E80A5AD43E3977ECDACF039AA04] every/(1/1) ...platform/0_VALUES_REPLACE
2016-01-14 17:16:40,106 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791799961_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/D0E4EE589CFB4119A517E1FC43A796C1/C7F31E80A5AD43E3977ECDACF039AA04/.tez/application_1452791799961_0001/localmode-log-dir/dag_1452791799961_0001_1.dot
2016-01-14 17:16:40,106 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/D0E4EE589CFB4119A517E1FC43A796C1/C7F31E80A5AD43E3977ECDACF039AA04/.tez/application_1452791799961_0001/localmode-log-dir/dag_1452791799961_0001_1-tez-dag.pb.txt
2016-01-14 17:16:40,112 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791799961_0001_1
2016-01-14 17:16:40,112 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:40,112 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [D0E4EE589CFB4119A517E1FC43A796C1/C7F31E80A5AD43E3977ECDACF039AA04] every/(1/1) ...platform/0_VALUES_REPLACE
2016-01-14 17:16:40 Running Dag: dag_1452791799961_0001_1
2016-01-14 17:16:40,123 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:40,124 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791799961_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791799961_0001_1, submitTime=1452791800105
2016-01-14 17:16:40,124 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex F30B0CB5E00442D79A2EE0479BA44784
2016-01-14 17:16:40,124 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:40,124 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:40,124 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791799961_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791799961_0001_1, initTime=1452791800124
2016-01-14 17:16:40,125 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791799961_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:40,125 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791799961_0001_1][Event:DAG_STARTED]: dagID=dag_1452791799961_0001_1, startTime=1452791800125
2016-01-14 17:16:40,125 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:40,125 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791799961_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:40,125 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: A1321A6B302E4EDEBC53788F7D9CD345 : {5A129CDDB4F74389BBBEF844670E265B={InputName=5A129CDDB4F74389BBBEF844670E265B}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:40,125 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 5A129CDDB4F74389BBBEF844670E265B, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:40,125 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791799961_0001_1_00 [A1321A6B302E4EDEBC53788F7D9CD345]
2016-01-14 17:16:40,125 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791799961_0001_1_00 [A1321A6B302E4EDEBC53788F7D9CD345]
2016-01-14 17:16:40,125 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791799961_0001_1_00 [A1321A6B302E4EDEBC53788F7D9CD345]
2016-01-14 17:16:40,125 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791799961_0001_1_00 [A1321A6B302E4EDEBC53788F7D9CD345]. Starting root input initializers: 1
2016-01-14 17:16:40,126 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:40,126 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791799961_0001_1_00 [A1321A6B302E4EDEBC53788F7D9CD345] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:40,126 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791799961_0001_1_01 [F30B0CB5E00442D79A2EE0479BA44784]
2016-01-14 17:16:40,126 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...platform/0_VALUES_REPLACE, applicationId=application_1452791799961_0001, dagName=[D0E4EE589CFB4119A517E1FC43A796C1/C7F31E80A5AD43E3977ECDACF039AA04] every/(1/1) ...platform/0_VALUES_REPLACE
2016-01-14 17:16:40,127 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 5A129CDDB4F74389BBBEF844670E265B on vertex vertex_1452791799961_0001_1_00 [A1321A6B302E4EDEBC53788F7D9CD345]
2016-01-14 17:16:40,129 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:40,129 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791799961_0001_1_01 [F30B0CB5E00442D79A2EE0479BA44784]
2016-01-14 17:16:40,129 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791799961_0001_1_01 [F30B0CB5E00442D79A2EE0479BA44784]
2016-01-14 17:16:40,129 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791799961_0001_1_01 [F30B0CB5E00442D79A2EE0479BA44784]
2016-01-14 17:16:40,129 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=8EB2D30F2E1D48A4A6F529762F34BAF3, vertexId=vertex_1452791799961_0001_1_01 [F30B0CB5E00442D79A2EE0479BA44784], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:40,129 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=8EB2D30F2E1D48A4A6F529762F34BAF3, vertexId=vertex_1452791799961_0001_1_01 [F30B0CB5E00442D79A2EE0479BA44784]
2016-01-14 17:16:40,131 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=8EB2D30F2E1D48A4A6F529762F34BAF3, vertexName=F30B0CB5E00442D79A2EE0479BA44784, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:40,131 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=8EB2D30F2E1D48A4A6F529762F34BAF3, vertexId=vertex_1452791799961_0001_1_01 [F30B0CB5E00442D79A2EE0479BA44784]
2016-01-14 17:16:40,132 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:40,133 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791799961_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F30B0CB5E00442D79A2EE0479BA44784, vertexId=vertex_1452791799961_0001_1_01, initRequestedTime=1452791800126, initedTime=1452791800132, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:40,133 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791799961_0001_1_01 [F30B0CB5E00442D79A2EE0479BA44784] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:40,133 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:40,134 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 5A129CDDB4F74389BBBEF844670E265B asking for 1 tasks. Headroom: 717 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:40,134 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:40,134 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:40,136 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:40,136 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 5A129CDDB4F74389BBBEF844670E265B on vertex vertex_1452791799961_0001_1_00 [A1321A6B302E4EDEBC53788F7D9CD345]
2016-01-14 17:16:40,137 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791799961_0001_1_00 [A1321A6B302E4EDEBC53788F7D9CD345] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:40,137 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {5A129CDDB4F74389BBBEF844670E265B=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:40,137 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791799961_0001_1_00 [A1321A6B302E4EDEBC53788F7D9CD345] parallelism set to 1
2016-01-14 17:16:40,137 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791799961_0001_1_00 [A1321A6B302E4EDEBC53788F7D9CD345] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:40,137 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791799961_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791799961_0001_1_00, eventCount=1
2016-01-14 17:16:40,138 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:40,138 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791799961_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A1321A6B302E4EDEBC53788F7D9CD345, vertexId=vertex_1452791799961_0001_1_00, initRequestedTime=1452791800125, initedTime=1452791800138, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:40,138 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791799961_0001_1_00 [A1321A6B302E4EDEBC53788F7D9CD345] with distanceFromRoot: 0
2016-01-14 17:16:40,138 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791799961_0001_1_00 [A1321A6B302E4EDEBC53788F7D9CD345] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:40,138 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in A1321A6B302E4EDEBC53788F7D9CD345
2016-01-14 17:16:40,139 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791799961_0001_1_00 [A1321A6B302E4EDEBC53788F7D9CD345]
2016-01-14 17:16:40,139 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:40,139 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791799961_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791799961_0001_1_00, startRequestedTime=1452791800138, startedTime=1452791800138
2016-01-14 17:16:40,139 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791799961_0001_1_00 [A1321A6B302E4EDEBC53788F7D9CD345] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:40,140 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:40,140 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791799961_0001_1][Event:TASK_STARTED]: vertexName=A1321A6B302E4EDEBC53788F7D9CD345, taskId=task_1452791799961_0001_1_00_000000, scheduledTime=1452791800140, launchTime=1452791800140
2016-01-14 17:16:40,140 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791799961_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:40,140 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791799961_0001_1_00 for vertex: vertex_1452791799961_0001_1_01 [F30B0CB5E00442D79A2EE0479BA44784] numStartedSources: 1 numSources: 1
2016-01-14 17:16:40,140 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791799961_0001_1_01 [F30B0CB5E00442D79A2EE0479BA44784] with distanceFromRoot: 1
2016-01-14 17:16:40,141 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791799961_0001_1_00 [A1321A6B302E4EDEBC53788F7D9CD345]
2016-01-14 17:16:40,141 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: A1321A6B302E4EDEBC53788F7D9CD345 in vertex: F30B0CB5E00442D79A2EE0479BA44784
2016-01-14 17:16:40,141 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: F30B0CB5E00442D79A2EE0479BA44784 with 1 source tasks and 1 pending tasks
2016-01-14 17:16:40,141 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:40,142 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791799961_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791799961_0001_1_01, startRequestedTime=1452791800141, startedTime=1452791800141
2016-01-14 17:16:40,142 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791799961_0001_1_01 [F30B0CB5E00442D79A2EE0479BA44784] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:40,142 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [D0E4EE589CFB4119A517E1FC43A796C1/C7F31E80A5AD43E3977ECDACF039AA04] every/(1/1) ...platform/0_VALUES_REPLACE, VertexName: A1321A6B302E4EDEBC53788F7D9CD345, VertexParallelism: 1, TaskAttemptID:attempt_1452791799961_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=5A129CDDB4F74389BBBEF844670E265B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F30B0CB5E00442D79A2EE0479BA44784, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:40,142 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791799961_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:40,142 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:40,143 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:40,143 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791799961_0001, with dag id: dag_1452791799961_0001_1
2016-01-14 17:16:40,143 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:40,143 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:40,144 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791799961_0001_00_000001
2016-01-14 17:16:40,144 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791799961_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:40,145 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791799961_0001_00_000001
2016-01-14 17:16:40,145 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:40,146 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791799961_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:40,146 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:40,146 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791799961_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791799961_0001_00_000001, launchTime=1452791800146
2016-01-14 17:16:40,146 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791799961_0001_00_000001
2016-01-14 17:16:40,147 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791799961_0001_1_00_000000_0] to container: [container_1452791799961_0001_00_000001]
2016-01-14 17:16:40,148 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791799961_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:40,148 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791799961_0001_1_00_000000_0] started. Is using containerId: [container_1452791799961_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:40,148 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791799961_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A1321A6B302E4EDEBC53788F7D9CD345, taskAttemptId=attempt_1452791799961_0001_1_00_000000_0, startTime=1452791800148, containerId=container_1452791799961_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791799961_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:40,148 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791799961_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:40,148 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791799961_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:40,149 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:40,149 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791799961_0001_00_000001 given task: attempt_1452791799961_0001_1_00_000000_0
2016-01-14 17:16:40,149 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791799961_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791799961_0001_1_00_000000_0
2016-01-14 17:16:40 Starting to run new task attempt: attempt_1452791799961_0001_1_00_000000_0
2016-01-14 17:16:40,149 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791799961_0001_1_00_000000_0
2016-01-14 17:16:40,149 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:40,150 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:40,150 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:40,150 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:40,150 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:40,150 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:40,150 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [D0E4EE589CFB4119A517E1FC43A796C1/C7F31E80A5AD43E3977ECDACF039AA04] every/(1/1) ...platform/0_VALUES_REPLACE, VertexName: A1321A6B302E4EDEBC53788F7D9CD345, VertexParallelism: 1, TaskAttemptID:attempt_1452791799961_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=5A129CDDB4F74389BBBEF844670E265B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F30B0CB5E00442D79A2EE0479BA44784, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:40,150 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=751828992
2016-01-14 17:16:40,151 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791799961_0001_1_00_000000_0
2016-01-14 17:16:40,151 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:40,151 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:40,151 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791799961_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:40,151 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=5A129CDDB4F74389BBBEF844670E265B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:40,152 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:40,152 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 5A129CDDB4F74389BBBEF844670E265B
2016-01-14 17:16:40,152 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:40,152 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F30B0CB5E00442D79A2EE0479BA44784, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:40,153 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:40,153 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F30B0CB5E00442D79A2EE0479BA44784
2016-01-14 17:16:40,154 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 5A129CDDB4F74389BBBEF844670E265B
2016-01-14 17:16:40,154 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:40,154 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: F30B0CB5E00442D79A2EE0479BA44784
2016-01-14 17:16:40,154 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F30B0CB5E00442D79A2EE0479BA44784
2016-01-14 17:16:40,155 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:40,156 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:40,157 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 5A129CDDB4F74389BBBEF844670E265B
2016-01-14 17:16:40,157 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 5A129CDDB4F74389BBBEF844670E265B
2016-01-14 17:16:40,158 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A1321A6B302E4EDEBC53788F7D9CD345, ordinal: 0
2016-01-14 17:16:40,158 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A1321A6B302E4EDEBC53788F7D9CD345, mem on start (mb), free: 300, total: 717, max: 717
2016-01-14 17:16:40,158 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:40,158 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:40,158 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:40,158 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:40,159 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:40,159 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:40,159 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:40,159 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:40,159 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:40,159 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 751828992, TotalAvailable: 503725424, TotalRequested/TotalJVMHeap:0.14
2016-01-14 17:16:40,159 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 5A129CDDB4F74389BBBEF844670E265B, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:40,159 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F30B0CB5E00442D79A2EE0479BA44784, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:40,160 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:40,160 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 5A129CDDB4F74389BBBEF844670E265B being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:40,160 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:40,160 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:40,160 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 5A129CDDB4F74389BBBEF844670E265B
2016-01-14 17:16:40,160 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:40,161 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 5A129CDDB4F74389BBBEF844670E265B
2016-01-14 17:16:40,161 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:40,161 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791799961_0001_1_00_000000_0
2016-01-14 17:16:40,161 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:40,162 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@58ed737c
2016-01-14 17:16:40,163 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@58ed737c
2016-01-14 17:16:40,163 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:40,163 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:40,164 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:40,164 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 5A129CDDB4F74389BBBEF844670E265B
2016-01-14 17:16:40,164 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 230F0D3879C6476B8086E55988264243
2016-01-14 17:16:40,165 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 230F0D3879C6476B8086E55988264243
2016-01-14 17:16:40,165 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@46f29a61; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@61757aaa; comparator=cascading.tuple.hadoop.util.TupleComparator@67c860e2; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:40,165 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:40,166 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:40,167 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:40,181 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:40,181 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:40,181 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:40,182 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:40,182 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:40,183 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:40,183 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:40,183 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A1321A6B302E4EDEBC53788F7D9CD345, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:40,186 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791799961_0001_1_00_000000_0
2016-01-14 17:16:40,186 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A1321A6B302E4EDEBC53788F7D9CD345, mem on close (mb), free: 199, total: 717, max: 717
2016-01-14 17:16:40,186 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:40,186 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:40,187 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:40,187 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:40,190 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:40 Completed running task attempt: attempt_1452791799961_0001_1_00_000000_0
2016-01-14 17:16:40,192 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791799961_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:40,193 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791799961_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791799961_0001_1_00, eventCount=1
2016-01-14 17:16:40,193 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:40,193 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791799961_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A1321A6B302E4EDEBC53788F7D9CD345, taskAttemptId=attempt_1452791799961_0001_1_00_000000_0, startTime=1452791800148, finishTime=1452791800193, timeTaken=45, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751828992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791800183, Process_Duration=3, Process_End_Time=1452791800186, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:40,193 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791799961_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:40,194 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:40,194 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791799961_0001_1_00_000000_0
2016-01-14 17:16:40,194 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791799961_0001_1][Event:TASK_FINISHED]: vertexName=A1321A6B302E4EDEBC53788F7D9CD345, taskId=task_1452791799961_0001_1_00_000000, startTime=1452791800148, finishTime=1452791800194, timeTaken=46, status=SUCCEEDED, successfulAttemptID=attempt_1452791799961_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751828992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791800183, Process_Duration=3, Process_End_Time=1452791800186, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:40,194 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:40,194 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791799961_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:40,195 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791799961_0001_1_00 [A1321A6B302E4EDEBC53788F7D9CD345] : 1
2016-01-14 17:16:40,195 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791799961_0001_1_00 [A1321A6B302E4EDEBC53788F7D9CD345]
2016-01-14 17:16:40,195 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:40,195 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791799961_0001_1][Event:VERTEX_FINISHED]: vertexName=A1321A6B302E4EDEBC53788F7D9CD345, vertexId=vertex_1452791799961_0001_1_00, initRequestedTime=1452791800125, initedTime=1452791800138, startRequestedTime=1452791800138, startedTime=1452791800138, finishTime=1452791800195, timeTaken=57, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751828992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791800183, Process_Duration=3, Process_End_Time=1452791800186, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791800148, firstTasksToStart=[ task_1452791799961_0001_1_00_000000 ], lastTaskFinishTime=1452791800193, lastTasksToFinish=[ task_1452791799961_0001_1_00_000000 ], minTaskDuration=45, maxTaskDuration=45, avgTaskDuration=45.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791799961_0001_1_00_000000 ], longestDurationTasks=[ task_1452791799961_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:40,195 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791799961_0001_1_00 [A1321A6B302E4EDEBC53788F7D9CD345] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:40,196 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791799961_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:40,196 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791799961_0001_1_01 [F30B0CB5E00442D79A2EE0479BA44784] attempt: attempt_1452791799961_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:40,196 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: F30B0CB5E00442D79A2EE0479BA44784
2016-01-14 17:16:40,196 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791799961_0001_1_01 [F30B0CB5E00442D79A2EE0479BA44784]
2016-01-14 17:16:40,196 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791799961_0001_1_00 [A1321A6B302E4EDEBC53788F7D9CD345] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:40,196 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791799961_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791799961_0001_00_000001, stoppedTime=1452791800196, exitStatus=0
2016-01-14 17:16:40,196 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791799961_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:40,197 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791799961_0001_1][Event:TASK_STARTED]: vertexName=F30B0CB5E00442D79A2EE0479BA44784, taskId=task_1452791799961_0001_1_01_000000, scheduledTime=1452791800197, launchTime=1452791800197
2016-01-14 17:16:40,197 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791799961_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:40,197 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791799961_0001_1_01 [F30B0CB5E00442D79A2EE0479BA44784]
2016-01-14 17:16:40,197 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [D0E4EE589CFB4119A517E1FC43A796C1/C7F31E80A5AD43E3977ECDACF039AA04] every/(1/1) ...platform/0_VALUES_REPLACE, VertexName: F30B0CB5E00442D79A2EE0479BA44784, VertexParallelism: 1, TaskAttemptID:attempt_1452791799961_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A1321A6B302E4EDEBC53788F7D9CD345, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=8EB2D30F2E1D48A4A6F529762F34BAF3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:40,197 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791799961_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:40,197 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:40,197 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751828992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791800183, Process_Duration=3, Process_End_Time=1452791800186, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:40,198 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791799961_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:40,198 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791799961_0001_00_000002
2016-01-14 17:16:40,198 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791799961_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:40,198 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791799961_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:40,198 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:40,198 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:40,200 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791799961_0001_00_000002
2016-01-14 17:16:40,200 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:40,200 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:40,200 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:40,201 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791799961_0001_00_000001
2016-01-14 17:16:40,201 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791799961_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:40,201 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791799961_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:40,201 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791799961_0001_00_000001, Exiting
2016-01-14 17:16:40,201 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791799961_0001_00_000001
2016-01-14 17:16:40,201 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791799961_0001_00_000001
2016-01-14 17:16:40,202 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791799961_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791799961_0001_00_000002, launchTime=1452791800200
2016-01-14 17:16:40,201 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791799961_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:40,201 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:40,204 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791799961_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:40,204 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791799961_0001_00_000001 completed successfully
2016-01-14 17:16:40,204 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791799961_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:40,204 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791799961_0001_00_000002
2016-01-14 17:16:40,205 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791799961_0001_1_01_000000_0] to container: [container_1452791799961_0001_00_000002]
2016-01-14 17:16:40,205 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791799961_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:40,206 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791799961_0001_1_01_000000_0] started. Is using containerId: [container_1452791799961_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:40,206 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791799961_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F30B0CB5E00442D79A2EE0479BA44784, taskAttemptId=attempt_1452791799961_0001_1_01_000000_0, startTime=1452791800206, containerId=container_1452791799961_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791799961_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:40,206 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791799961_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:40,206 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791799961_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:40,206 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:40,206 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791799961_0001_00_000002 given task: attempt_1452791799961_0001_1_01_000000_0
2016-01-14 17:16:40,207 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791799961_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791799961_0001_1_01_000000_0
2016-01-14 17:16:40 Starting to run new task attempt: attempt_1452791799961_0001_1_01_000000_0
2016-01-14 17:16:40,208 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791799961_0001_1_01_000000_0
2016-01-14 17:16:40,208 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:40,208 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:40,208 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:40,208 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:40,208 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:40,208 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:40,208 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [D0E4EE589CFB4119A517E1FC43A796C1/C7F31E80A5AD43E3977ECDACF039AA04] every/(1/1) ...platform/0_VALUES_REPLACE, VertexName: F30B0CB5E00442D79A2EE0479BA44784, VertexParallelism: 1, TaskAttemptID:attempt_1452791799961_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A1321A6B302E4EDEBC53788F7D9CD345, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=8EB2D30F2E1D48A4A6F529762F34BAF3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:40,208 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=751828992
2016-01-14 17:16:40,209 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791799961_0001_1_01_000000_0
2016-01-14 17:16:40,209 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:40,210 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:40,210 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:40,210 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=A1321A6B302E4EDEBC53788F7D9CD345, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:40,210 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:40,211 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: A1321A6B302E4EDEBC53788F7D9CD345
2016-01-14 17:16:40,212 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 676646080, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:40,212 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:40,212 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 676646080, type: INPUT, componentVertexName: A1321A6B302E4EDEBC53788F7D9CD345
2016-01-14 17:16:40,212 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: A1321A6B302E4EDEBC53788F7D9CD345
2016-01-14 17:16:40,213 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791799961_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:40,213 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:40,217 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=8EB2D30F2E1D48A4A6F529762F34BAF3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:40,217 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:40,217 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 8EB2D30F2E1D48A4A6F529762F34BAF3
2016-01-14 17:16:40,217 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:40,218 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 8EB2D30F2E1D48A4A6F529762F34BAF3
2016-01-14 17:16:40,219 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F30B0CB5E00442D79A2EE0479BA44784, ordinal: 1
2016-01-14 17:16:40,219 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F30B0CB5E00442D79A2EE0479BA44784, mem on start (mb), free: 194, total: 717, max: 717
2016-01-14 17:16:40,219 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:40,219 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:40,219 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:40,227 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:40,227 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 8EB2D30F2E1D48A4A6F529762F34BAF3
2016-01-14 17:16:40,227 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:40,227 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:40,227 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:40,227 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:40,227 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:40,227 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:40,228 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 676646080, TotalRequestedScaled: 6.245963815384616E8, TotalJVMHeap: 751828992, TotalAvailable: 503725424, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:40,228 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, A1321A6B302E4EDEBC53788F7D9CD345, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=676646080, allocated=503725424
2016-01-14 17:16:40,228 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 8EB2D30F2E1D48A4A6F529762F34BAF3, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:40,228 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:40,228 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: A1321A6B302E4EDEBC53788F7D9CD345 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:40,228 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:40,228 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:40,228 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: A1321A6B302E4EDEBC53788F7D9CD345
2016-01-14 17:16:40,229 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:40,229 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:40,229 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: A1321A6B302E4EDEBC53788F7D9CD345 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:40,229 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=676646080, postMergeMem=0, RuntimeTotalAvailable=503725424. Updated to: ShuffleMem=503725424, postMergeMem: 0
2016-01-14 17:16:40,229 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=503725424, maxSingleShuffleLimit=125931360, mergeThreshold=453352896, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:40,229 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:40,230 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: A1321A6B302E4EDEBC53788F7D9CD345
2016-01-14 17:16:40,230 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:40,230 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791799961_0001_1_01_000000_0
2016-01-14 17:16:40,232 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791799961_0001_1_00_000000_0_10195, runDuration: 38518, ]
2016-01-14 17:16:40,233 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791799961_0001_1_00_000000_0_10195] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791800233, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 17:16:40,233 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : A1321A6B302E4EDEBC53788F7D9CD345
2016-01-14 17:16:40,233 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:40,233 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [A1321A6B302E4EDEBC53788F7D9CD345] #33 in 1ms
2016-01-14 17:16:40,233 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:40,233 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [A1321A6B302E4EDEBC53788F7D9CD345] #33, status:false, isInterrupted:false
2016-01-14 17:16:40,234 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:40,234 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 230F0D3879C6476B8086E55988264243
2016-01-14 17:16:40,234 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:40,234 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_REPLACE"], id: 8EB2D30F2E1D48A4A6F529762F34BAF3
2016-01-14 17:16:40,234 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_REPLACE"]
2016-01-14 17:16:40,234 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:40,234 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:40,234 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 230F0D3879C6476B8086E55988264243, for 1 inputs
2016-01-14 17:16:40,235 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:40,235 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : A1321A6B302E4EDEBC53788F7D9CD345
2016-01-14 17:16:40,235 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:40,235 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F30B0CB5E00442D79A2EE0479BA44784, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:40,235 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@2c28a5ec; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@20a53f9a
2016-01-14 17:16:40,236 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:40,237 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(92)) - caught throwable
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:40,238 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:40,238 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:40,239 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791799961_0001_1_01_000000_0
2016-01-14 17:16:40,239 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791799961_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:40,239 INFO  task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1452791799961_0001_1_01_000000_0
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:40,240 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751828992, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791800235, Process_Duration=4, Process_End_Time=1452791800239, Read_Duration=0, Tuples_Read=1]]
2016-01-14 17:16:40,240 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791799961_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F30B0CB5E00442D79A2EE0479BA44784, taskAttemptId=attempt_1452791799961_0001_1_01_000000_0, startTime=1452791800206, finishTime=1452791800240, timeTaken=34, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751828992, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791800235, Process_Duration=4, Process_End_Time=1452791800239, Read_Duration=0, Tuples_Read=1
2016-01-14 17:16:40,241 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791799961_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-01-14 17:16:40,241 INFO  impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1452791799961_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-01-14 17:16:40,240 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:40,241 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791799961_0001_1][Event:TASK_FINISHED]: vertexName=F30B0CB5E00442D79A2EE0479BA44784, taskId=task_1452791799961_0001_1_01_000000, startTime=1452791800206, finishTime=1452791800241, timeTaken=35, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
], counters=Counters: 0
2016-01-14 17:16:40,241 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791799961_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-01-14 17:16:40,241 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:40,241 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:40,241 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F30B0CB5E00442D79A2EE0479BA44784, mem on close (mb), free: 191, total: 717, max: 717
2016-01-14 17:16:40,241 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791799961_0001_1_01 [F30B0CB5E00442D79A2EE0479BA44784] : 1
2016-01-14 17:16:40,242 INFO  impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1452791799961_0001_1_01 [F30B0CB5E00442D79A2EE0479BA44784] because task failed: task_1452791799961_0001_1_01_000000
2016-01-14 17:16:40,242 INFO  impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1452791799961_0001_1_01 [F30B0CB5E00442D79A2EE0479BA44784] due to trigger: OWN_TASK_FAILURE
2016-01-14 17:16:40,242 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-01-14 17:16:40,242 INFO  impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1452791799961_0001_1_01 [F30B0CB5E00442D79A2EE0479BA44784]
2016-01-14 17:16:40,243 INFO  impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=8EB2D30F2E1D48A4A6F529762F34BAF3, vertexId=vertex_1452791799961_0001_1_01 [F30B0CB5E00442D79A2EE0479BA44784]
2016-01-14 17:16:40,243 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=F30B0CB5E00442D79A2EE0479BA44784, index=1
2016-01-14 17:16:40,244 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: A1321A6B302E4EDEBC53788F7D9CD345
2016-01-14 17:16:40,243 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791799961_0001_1][Event:VERTEX_FINISHED]: vertexName=F30B0CB5E00442D79A2EE0479BA44784, vertexId=vertex_1452791799961_0001_1_01, initRequestedTime=1452791800126, initedTime=1452791800132, startRequestedTime=1452791800141, startedTime=1452791800141, finishTime=1452791800243, timeTaken=102, status=FAILED, diagnostics=Task failed, taskId=task_1452791799961_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1452791799961_0001_1_01 [F30B0CB5E00442D79A2EE0479BA44784] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1452791800206, firstTasksToStart=[ task_1452791799961_0001_1_01_000000 ], lastTaskFinishTime=1452791800240, lastTasksToFinish=[ task_1452791799961_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[  ], longestDurationTasks=[  ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-01-14 17:16:40,244 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791799961_0001_1_01 [F30B0CB5E00442D79A2EE0479BA44784] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-01-14 17:16:40,244 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791799961_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791799961_0001_00_000002, stoppedTime=1452791800244, exitStatus=0
2016-01-14 17:16:40,243 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:40,244 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791799961_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:40,245 INFO  node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1452791799961_0001_1_01_000000_0 failed: true container: container_1452791799961_0001_00_000002 numFailedTAs: 0
2016-01-14 17:16:40,245 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:40,244 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:40,245 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791799961_0001_1_01 [F30B0CB5E00442D79A2EE0479BA44784] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-01-14 17:16:40,245 INFO  impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-01-14 17:16:40,245 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791799961_0001_1
2016-01-14 17:16:40,245 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 8EB2D30F2E1D48A4A6F529762F34BAF3 for vertex: vertex_1452791799961_0001_1_01 [F30B0CB5E00442D79A2EE0479BA44784]
2016-01-14 17:16:40,245 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1452791799961_0001_1_00 [A1321A6B302E4EDEBC53788F7D9CD345]
2016-01-14 17:16:40,246 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791799961_0001_1, queueSize=0
2016-01-14 17:16:40,246 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:40,246 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:40,247 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:40,247 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:40,247 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:40,247 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=F30B0CB5E00442D79A2EE0479BA44784, sourceVertex=A1321A6B302E4EDEBC53788F7D9CD345
2016-01-14 17:16:40,247 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791799961_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791799961_0001_1, startTime=1452791800125, finishTime=1452791800245, timeTaken=120, status=FAILED, diagnostics=Vertex failed, vertexName=F30B0CB5E00442D79A2EE0479BA44784, vertexId=vertex_1452791799961_0001_1_01, diagnostics=[Task failed, taskId=task_1452791799961_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1452791799961_0001_1_01 [F30B0CB5E00442D79A2EE0479BA44784] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751828992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791800183, Process_Duration=3, Process_End_Time=1452791800186, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:40,247 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791799961_0001_1 finished with state: FAILED
2016-01-14 17:16:40,248 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791799961_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:40,248 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791799961_0001_00_000002. Relying on regular task shutdown for it to end
2016-01-14 17:16:40,247 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=F30B0CB5E00442D79A2EE0479BA44784, sourceVertex=8EB2D30F2E1D48A4A6F529762F34BAF3
2016-01-14 17:16:40 Completed Dag: dag_1452791799961_0001_1
2016-01-14 17:16:40,248 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791799961_0001_1, dagState=FAILED
2016-01-14 17:16:40,248 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791799961_0001_00_000002
2016-01-14 17:16:40,248 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791799961_0001_00_000002
2016-01-14 17:16:40,248 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791799961_0001_1_post
2016-01-14 17:16:40,249 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791799961_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-01-14 17:16:40,249 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1452791799961_0001_00_000002 completed but with errors
2016-01-14 17:16:40,249 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:40,249 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:40,249 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791799961_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:40,249 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1452791799961_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-01-14 17:16:40,249 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791799961_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:40,249 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791799961_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-01-14 17:16:40,255 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...platform/0_VALUES_REPLACE, applicationId=application_1452791799961_0001
2016-01-14 17:16:40,256 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:40,256 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:40,257 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:40,257 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:40,257 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:40,257 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:40,257 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:40,257 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:40,257 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:40,258 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:40,258 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:40,258 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:40,258 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:40,263 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/D0E4EE589CFB4119A517E1FC43A796C1/C7F31E80A5AD43E3977ECDACF039AA04/.tez/application_1452791799961_0001
2016-01-14 17:16:40,263 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping all jobs
2016-01-14 17:16:40,264 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping: (1/1) ...platform/0_VALUES_REPLACE
2016-01-14 17:16:40,264 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopped all jobs
2016-01-14 17:16:40,265 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.321
2016-01-14 17:16:40,265 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_VALUES_REPLACE
2016-01-14 17:16:40,295 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:40,296 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:40,353 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.008
2016-01-14 17:16:40,357 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-01-14 17:16:40,358 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:40,358 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:40,358 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:40,372 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:40,377 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_SWAP
2016-01-14 17:16:40,420 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:40,421 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:40,421 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_SWAP"]
2016-01-14 17:16:40,421 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:40,421 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:40,421 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:40,422 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...blyplatform/0_VALUES_SWAP
2016-01-14 17:16:40,422 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:40,434 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:40,440 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:40,440 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:40,440 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:40,449 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/D489E4664F28487E8AEA666FA545563B/C49AE1AC88EF4C57BA25A5A085556181/.tez/application_1452791800440_0001 doesn't exist and is created
2016-01-14 17:16:40,464 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:40,465 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:40,465 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/D489E4664F28487E8AEA666FA545563B/C49AE1AC88EF4C57BA25A5A085556181/.tez/application_1452791800440_0001
2016-01-14 17:16:40,468 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791800440_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:40,478 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:40,479 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:40,480 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:40,480 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:40,480 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/D489E4664F28487E8AEA666FA545563B/C49AE1AC88EF4C57BA25A5A085556181/.tez/application_1452791800440_0001/localmode-log-dir
2016-01-14 17:16:40,480 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/D489E4664F28487E8AEA666FA545563B/C49AE1AC88EF4C57BA25A5A085556181/.tez/application_1452791800440_0001/localmode-log-dir/history.txt.appattempt_1452791800440_0001_000000, maxErrors=10
2016-01-14 17:16:40,480 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:40,481 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791800440_0001_000000, appSubmitTime=1452791800468, launchTime=1452791800468
2016-01-14 17:16:40,484 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:40,487 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:40,488 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:40,488 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:40,489 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:59898
2016-01-14 17:16:40,491 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:40,492 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791800440_0001_000000, startTime=1452791800492
2016-01-14 17:16:40,492 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:40,492 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:40,495 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 17:16:40,495 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 17:16:40,565 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:40,565 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:40,565 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...blyplatform/0_VALUES_SWAP, applicationId=application_1452791800440_0001, dagName=[D489E4664F28487E8AEA666FA545563B/C49AE1AC88EF4C57BA25A5A085556181] every/(1/1) ...blyplatform/0_VALUES_SWAP
2016-01-14 17:16:40,581 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [D489E4664F28487E8AEA666FA545563B/C49AE1AC88EF4C57BA25A5A085556181] every/(1/1) ...blyplatform/0_VALUES_SWAP
2016-01-14 17:16:40,582 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791800440_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/D489E4664F28487E8AEA666FA545563B/C49AE1AC88EF4C57BA25A5A085556181/.tez/application_1452791800440_0001/localmode-log-dir/dag_1452791800440_0001_1.dot
2016-01-14 17:16:40,583 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/D489E4664F28487E8AEA666FA545563B/C49AE1AC88EF4C57BA25A5A085556181/.tez/application_1452791800440_0001/localmode-log-dir/dag_1452791800440_0001_1-tez-dag.pb.txt
2016-01-14 17:16:40,588 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791800440_0001_1
2016-01-14 17:16:40,589 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:40,589 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [D489E4664F28487E8AEA666FA545563B/C49AE1AC88EF4C57BA25A5A085556181] every/(1/1) ...blyplatform/0_VALUES_SWAP
2016-01-14 17:16:40 Running Dag: dag_1452791800440_0001_1
2016-01-14 17:16:40,600 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:40,600 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791800440_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791800440_0001_1, submitTime=1452791800581
2016-01-14 17:16:40,601 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex D29E7753543149C8A034F79792BF05F2
2016-01-14 17:16:40,601 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:40,601 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:40,602 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791800440_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791800440_0001_1, initTime=1452791800600
2016-01-14 17:16:40,602 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791800440_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:40,602 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791800440_0001_1][Event:DAG_STARTED]: dagID=dag_1452791800440_0001_1, startTime=1452791800602
2016-01-14 17:16:40,602 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:40,602 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:40,602 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791800440_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:40,602 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: F63C3F458E2B423A9F0C681C9129FA12 : {1A20BA00A1864FBEAE32DC0DF0B49113={InputName=1A20BA00A1864FBEAE32DC0DF0B49113}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:40,602 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 1A20BA00A1864FBEAE32DC0DF0B49113, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:40,603 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791800440_0001_1_00 [F63C3F458E2B423A9F0C681C9129FA12]
2016-01-14 17:16:40,603 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...blyplatform/0_VALUES_SWAP, applicationId=application_1452791800440_0001, dagName=[D489E4664F28487E8AEA666FA545563B/C49AE1AC88EF4C57BA25A5A085556181] every/(1/1) ...blyplatform/0_VALUES_SWAP
2016-01-14 17:16:40,603 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791800440_0001_1_00 [F63C3F458E2B423A9F0C681C9129FA12]
2016-01-14 17:16:40,603 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791800440_0001_1_00 [F63C3F458E2B423A9F0C681C9129FA12]
2016-01-14 17:16:40,603 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791800440_0001_1_00 [F63C3F458E2B423A9F0C681C9129FA12]. Starting root input initializers: 1
2016-01-14 17:16:40,603 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791800440_0001_1_00 [F63C3F458E2B423A9F0C681C9129FA12] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:40,603 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791800440_0001_1_01 [D29E7753543149C8A034F79792BF05F2]
2016-01-14 17:16:40,605 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 1A20BA00A1864FBEAE32DC0DF0B49113 on vertex vertex_1452791800440_0001_1_00 [F63C3F458E2B423A9F0C681C9129FA12]
2016-01-14 17:16:40,606 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:40,606 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791800440_0001_1_01 [D29E7753543149C8A034F79792BF05F2]
2016-01-14 17:16:40,606 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791800440_0001_1_01 [D29E7753543149C8A034F79792BF05F2]
2016-01-14 17:16:40,606 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791800440_0001_1_01 [D29E7753543149C8A034F79792BF05F2]
2016-01-14 17:16:40,606 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=6A76302C3BFA4EC0ABB0204D19EB3F8A, vertexId=vertex_1452791800440_0001_1_01 [D29E7753543149C8A034F79792BF05F2], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:40,606 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=6A76302C3BFA4EC0ABB0204D19EB3F8A, vertexId=vertex_1452791800440_0001_1_01 [D29E7753543149C8A034F79792BF05F2]
2016-01-14 17:16:40,607 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:40,607 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 1A20BA00A1864FBEAE32DC0DF0B49113 asking for 1 tasks. Headroom: 717 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:40,607 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:40,607 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:40,608 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:40,609 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=6A76302C3BFA4EC0ABB0204D19EB3F8A, vertexName=D29E7753543149C8A034F79792BF05F2, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:40,609 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 1A20BA00A1864FBEAE32DC0DF0B49113 on vertex vertex_1452791800440_0001_1_00 [F63C3F458E2B423A9F0C681C9129FA12]
2016-01-14 17:16:40,609 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=6A76302C3BFA4EC0ABB0204D19EB3F8A, vertexId=vertex_1452791800440_0001_1_01 [D29E7753543149C8A034F79792BF05F2]
2016-01-14 17:16:40,610 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:40,610 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791800440_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D29E7753543149C8A034F79792BF05F2, vertexId=vertex_1452791800440_0001_1_01, initRequestedTime=1452791800603, initedTime=1452791800610, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:40,610 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791800440_0001_1_01 [D29E7753543149C8A034F79792BF05F2] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:40,610 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791800440_0001_1_00 [F63C3F458E2B423A9F0C681C9129FA12] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:40,611 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {1A20BA00A1864FBEAE32DC0DF0B49113=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:40,611 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791800440_0001_1_00 [F63C3F458E2B423A9F0C681C9129FA12] parallelism set to 1
2016-01-14 17:16:40,611 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791800440_0001_1_00 [F63C3F458E2B423A9F0C681C9129FA12] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:40,611 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791800440_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791800440_0001_1_00, eventCount=1
2016-01-14 17:16:40,612 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:40,612 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791800440_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F63C3F458E2B423A9F0C681C9129FA12, vertexId=vertex_1452791800440_0001_1_00, initRequestedTime=1452791800602, initedTime=1452791800612, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:40,612 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791800440_0001_1_00 [F63C3F458E2B423A9F0C681C9129FA12] with distanceFromRoot: 0
2016-01-14 17:16:40,612 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791800440_0001_1_00 [F63C3F458E2B423A9F0C681C9129FA12] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:40,612 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in F63C3F458E2B423A9F0C681C9129FA12
2016-01-14 17:16:40,613 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791800440_0001_1_00 [F63C3F458E2B423A9F0C681C9129FA12]
2016-01-14 17:16:40,613 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:40,613 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791800440_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791800440_0001_1_00, startRequestedTime=1452791800612, startedTime=1452791800612
2016-01-14 17:16:40,613 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791800440_0001_1_00 [F63C3F458E2B423A9F0C681C9129FA12] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:40,613 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:40,614 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791800440_0001_1][Event:TASK_STARTED]: vertexName=F63C3F458E2B423A9F0C681C9129FA12, taskId=task_1452791800440_0001_1_00_000000, scheduledTime=1452791800613, launchTime=1452791800613
2016-01-14 17:16:40,614 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791800440_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:40,614 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791800440_0001_1_00 for vertex: vertex_1452791800440_0001_1_01 [D29E7753543149C8A034F79792BF05F2] numStartedSources: 1 numSources: 1
2016-01-14 17:16:40,614 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791800440_0001_1_01 [D29E7753543149C8A034F79792BF05F2] with distanceFromRoot: 1
2016-01-14 17:16:40,614 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791800440_0001_1_00 [F63C3F458E2B423A9F0C681C9129FA12]
2016-01-14 17:16:40,615 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: F63C3F458E2B423A9F0C681C9129FA12 in vertex: D29E7753543149C8A034F79792BF05F2
2016-01-14 17:16:40,615 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: D29E7753543149C8A034F79792BF05F2 with 1 source tasks and 1 pending tasks
2016-01-14 17:16:40,615 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:40,615 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791800440_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791800440_0001_1_01, startRequestedTime=1452791800615, startedTime=1452791800615
2016-01-14 17:16:40,616 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791800440_0001_1_01 [D29E7753543149C8A034F79792BF05F2] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:40,616 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [D489E4664F28487E8AEA666FA545563B/C49AE1AC88EF4C57BA25A5A085556181] every/(1/1) ...blyplatform/0_VALUES_SWAP, VertexName: F63C3F458E2B423A9F0C681C9129FA12, VertexParallelism: 1, TaskAttemptID:attempt_1452791800440_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=1A20BA00A1864FBEAE32DC0DF0B49113, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D29E7753543149C8A034F79792BF05F2, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:40,616 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791800440_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:40,616 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:40,617 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:40,617 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:40,617 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:40,618 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791800440_0001_00_000001
2016-01-14 17:16:40,618 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791800440_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:40,619 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791800440_0001_00_000001
2016-01-14 17:16:40,619 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:40,620 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791800440_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:40,620 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791800440_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791800440_0001_00_000001, launchTime=1452791800620
2016-01-14 17:16:40,620 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:40,620 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791800440_0001_00_000001
2016-01-14 17:16:40,620 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791800440_0001_1_00_000000_0] to container: [container_1452791800440_0001_00_000001]
2016-01-14 17:16:40,621 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791800440_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:40,621 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791800440_0001_00_000001 given task: attempt_1452791800440_0001_1_00_000000_0
2016-01-14 17:16:40,621 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791800440_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791800440_0001_1_00_000000_0
2016-01-14 17:16:40 Starting to run new task attempt: attempt_1452791800440_0001_1_00_000000_0
2016-01-14 17:16:40,621 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791800440_0001_1_00_000000_0] started. Is using containerId: [container_1452791800440_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:40,621 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791800440_0001_1_00_000000_0
2016-01-14 17:16:40,622 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:40,622 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:40,622 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:40,622 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:40,622 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:40,622 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791800440_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F63C3F458E2B423A9F0C681C9129FA12, taskAttemptId=attempt_1452791800440_0001_1_00_000000_0, startTime=1452791800621, containerId=container_1452791800440_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791800440_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:40,622 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:40,622 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:40,622 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [D489E4664F28487E8AEA666FA545563B/C49AE1AC88EF4C57BA25A5A085556181] every/(1/1) ...blyplatform/0_VALUES_SWAP, VertexName: F63C3F458E2B423A9F0C681C9129FA12, VertexParallelism: 1, TaskAttemptID:attempt_1452791800440_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=1A20BA00A1864FBEAE32DC0DF0B49113, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D29E7753543149C8A034F79792BF05F2, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:40,623 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=751828992
2016-01-14 17:16:40,623 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791800440_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:40,623 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791800440_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:40,624 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791800440_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:40,624 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791800440_0001_1_00_000000_0
2016-01-14 17:16:40,624 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:40,625 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:40,625 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791800440_0001, with dag id: dag_1452791800440_0001_1
2016-01-14 17:16:40,625 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:40,625 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=D29E7753543149C8A034F79792BF05F2, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:40,626 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:40,626 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: D29E7753543149C8A034F79792BF05F2
2016-01-14 17:16:40,627 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=1A20BA00A1864FBEAE32DC0DF0B49113, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:40,628 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:40,628 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 1A20BA00A1864FBEAE32DC0DF0B49113
2016-01-14 17:16:40,628 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 1A20BA00A1864FBEAE32DC0DF0B49113
2016-01-14 17:16:40,628 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:40,628 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:40,630 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: D29E7753543149C8A034F79792BF05F2
2016-01-14 17:16:40,631 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: D29E7753543149C8A034F79792BF05F2
2016-01-14 17:16:40,631 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:40,632 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 1A20BA00A1864FBEAE32DC0DF0B49113
2016-01-14 17:16:40,632 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 1A20BA00A1864FBEAE32DC0DF0B49113
2016-01-14 17:16:40,634 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F63C3F458E2B423A9F0C681C9129FA12, ordinal: 0
2016-01-14 17:16:40,634 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F63C3F458E2B423A9F0C681C9129FA12, mem on start (mb), free: 313, total: 717, max: 717
2016-01-14 17:16:40,634 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:40,634 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:40,634 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:40,634 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:40,634 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:40,635 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:40,635 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:40,635 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:40,635 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:40,635 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 751828992, TotalAvailable: 503725424, TotalRequested/TotalJVMHeap:0.14
2016-01-14 17:16:40,635 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 1A20BA00A1864FBEAE32DC0DF0B49113, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:40,635 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, D29E7753543149C8A034F79792BF05F2, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:40,635 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:40,635 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 1A20BA00A1864FBEAE32DC0DF0B49113 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:40,635 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:40,636 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:40,636 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 1A20BA00A1864FBEAE32DC0DF0B49113
2016-01-14 17:16:40,636 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:40,636 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 1A20BA00A1864FBEAE32DC0DF0B49113
2016-01-14 17:16:40,636 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:40,636 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791800440_0001_1_00_000000_0
2016-01-14 17:16:40,639 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:40,642 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:40,643 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 1A20BA00A1864FBEAE32DC0DF0B49113
2016-01-14 17:16:40,643 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: B04DE700A9044A75B75F2D3CA7269B64
2016-01-14 17:16:40,643 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] B04DE700A9044A75B75F2D3CA7269B64
2016-01-14 17:16:40,643 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@461cd6bf; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@f56b98f; comparator=cascading.tuple.hadoop.util.TupleComparator@6b06327; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:40,644 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:40,644 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:40,644 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:40,644 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@263d98df
2016-01-14 17:16:40,647 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@263d98df
2016-01-14 17:16:40,647 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:40,647 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:40,656 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:40,657 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:40,657 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:40,657 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:40,657 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:40,659 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:40,659 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:40,659 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F63C3F458E2B423A9F0C681C9129FA12, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:40,661 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791800440_0001_1_00_000000_0
2016-01-14 17:16:40,661 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F63C3F458E2B423A9F0C681C9129FA12, mem on close (mb), free: 211, total: 717, max: 717
2016-01-14 17:16:40,661 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:40,662 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:40,662 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:40,662 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:40,664 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:40 Completed running task attempt: attempt_1452791800440_0001_1_00_000000_0
2016-01-14 17:16:40,666 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791800440_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:40,666 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791800440_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791800440_0001_1_00, eventCount=1
2016-01-14 17:16:40,667 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:40,667 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791800440_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F63C3F458E2B423A9F0C681C9129FA12, taskAttemptId=attempt_1452791800440_0001_1_00_000000_0, startTime=1452791800621, finishTime=1452791800667, timeTaken=46, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751828992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791800659, Process_Duration=2, Process_End_Time=1452791800661, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:40,667 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791800440_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:40,667 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791800440_0001_1_00_000000_0
2016-01-14 17:16:40,667 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:40,667 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791800440_0001_1][Event:TASK_FINISHED]: vertexName=F63C3F458E2B423A9F0C681C9129FA12, taskId=task_1452791800440_0001_1_00_000000, startTime=1452791800621, finishTime=1452791800667, timeTaken=46, status=SUCCEEDED, successfulAttemptID=attempt_1452791800440_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751828992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791800659, Process_Duration=2, Process_End_Time=1452791800661, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:40,667 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791800440_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:40,667 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791800440_0001_1_00 [F63C3F458E2B423A9F0C681C9129FA12] : 1
2016-01-14 17:16:40,667 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791800440_0001_1_00 [F63C3F458E2B423A9F0C681C9129FA12]
2016-01-14 17:16:40,669 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:40,670 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791800440_0001_1][Event:VERTEX_FINISHED]: vertexName=F63C3F458E2B423A9F0C681C9129FA12, vertexId=vertex_1452791800440_0001_1_00, initRequestedTime=1452791800602, initedTime=1452791800612, startRequestedTime=1452791800612, startedTime=1452791800612, finishTime=1452791800668, timeTaken=56, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751828992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791800659, Process_Duration=2, Process_End_Time=1452791800661, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791800621, firstTasksToStart=[ task_1452791800440_0001_1_00_000000 ], lastTaskFinishTime=1452791800667, lastTasksToFinish=[ task_1452791800440_0001_1_00_000000 ], minTaskDuration=46, maxTaskDuration=46, avgTaskDuration=46.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791800440_0001_1_00_000000 ], longestDurationTasks=[ task_1452791800440_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:40,670 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791800440_0001_1_00 [F63C3F458E2B423A9F0C681C9129FA12] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:40,671 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791800440_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:40,672 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791800440_0001_1_01 [D29E7753543149C8A034F79792BF05F2] attempt: attempt_1452791800440_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:40,672 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: D29E7753543149C8A034F79792BF05F2
2016-01-14 17:16:40,672 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791800440_0001_1_01 [D29E7753543149C8A034F79792BF05F2]
2016-01-14 17:16:40,672 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791800440_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791800440_0001_00_000001, stoppedTime=1452791800672, exitStatus=0
2016-01-14 17:16:40,672 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791800440_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:40,672 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791800440_0001_1_00 [F63C3F458E2B423A9F0C681C9129FA12] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:40,673 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791800440_0001_1][Event:TASK_STARTED]: vertexName=D29E7753543149C8A034F79792BF05F2, taskId=task_1452791800440_0001_1_01_000000, scheduledTime=1452791800673, launchTime=1452791800673
2016-01-14 17:16:40,673 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791800440_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:40,673 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791800440_0001_1_01 [D29E7753543149C8A034F79792BF05F2]
2016-01-14 17:16:40,673 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [D489E4664F28487E8AEA666FA545563B/C49AE1AC88EF4C57BA25A5A085556181] every/(1/1) ...blyplatform/0_VALUES_SWAP, VertexName: D29E7753543149C8A034F79792BF05F2, VertexParallelism: 1, TaskAttemptID:attempt_1452791800440_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F63C3F458E2B423A9F0C681C9129FA12, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=6A76302C3BFA4EC0ABB0204D19EB3F8A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:40,673 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791800440_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:40,673 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:40,674 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:40,673 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751828992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791800659, Process_Duration=2, Process_End_Time=1452791800661, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:40,673 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791800440_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:40,674 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:40,675 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:40,674 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:40,674 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791800440_0001_00_000002
2016-01-14 17:16:40,675 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:40,675 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791800440_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:40,675 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791800440_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:40,677 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791800440_0001_00_000001
2016-01-14 17:16:40,677 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791800440_0001_00_000002
2016-01-14 17:16:40,677 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791800440_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:40,677 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:40,677 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791800440_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:40,677 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791800440_0001_00_000001, Exiting
2016-01-14 17:16:40,677 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791800440_0001_00_000001
2016-01-14 17:16:40,677 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791800440_0001_00_000001
2016-01-14 17:16:40,678 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791800440_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:40,678 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:40,678 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791800440_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791800440_0001_00_000002, launchTime=1452791800678
2016-01-14 17:16:40,679 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791800440_0001_00_000002
2016-01-14 17:16:40,679 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791800440_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:40,679 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791800440_0001_00_000001 completed successfully
2016-01-14 17:16:40,679 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791800440_0001_1_01_000000_0] to container: [container_1452791800440_0001_00_000002]
2016-01-14 17:16:40,679 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791800440_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:40,679 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791800440_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:40,680 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791800440_0001_00_000002 given task: attempt_1452791800440_0001_1_01_000000_0
2016-01-14 17:16:40,680 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791800440_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791800440_0001_1_01_000000_0
2016-01-14 17:16:40,680 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791800440_0001_1_01_000000_0] started. Is using containerId: [container_1452791800440_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:40,680 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:40 Starting to run new task attempt: attempt_1452791800440_0001_1_01_000000_0
2016-01-14 17:16:40,680 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791800440_0001_1_01_000000_0
2016-01-14 17:16:40,680 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:40,680 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791800440_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D29E7753543149C8A034F79792BF05F2, taskAttemptId=attempt_1452791800440_0001_1_01_000000_0, startTime=1452791800680, containerId=container_1452791800440_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791800440_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:40,681 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791800440_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:40,681 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791800440_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:40,681 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:40,681 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:40,681 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:40,681 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:40,681 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:40,681 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [D489E4664F28487E8AEA666FA545563B/C49AE1AC88EF4C57BA25A5A085556181] every/(1/1) ...blyplatform/0_VALUES_SWAP, VertexName: D29E7753543149C8A034F79792BF05F2, VertexParallelism: 1, TaskAttemptID:attempt_1452791800440_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F63C3F458E2B423A9F0C681C9129FA12, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=6A76302C3BFA4EC0ABB0204D19EB3F8A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:40,681 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=751828992
2016-01-14 17:16:40,682 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791800440_0001_1_01_000000_0
2016-01-14 17:16:40,682 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:40,682 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:40,682 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:40,683 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F63C3F458E2B423A9F0C681C9129FA12, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:40,683 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791800440_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:40,684 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:40,684 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F63C3F458E2B423A9F0C681C9129FA12
2016-01-14 17:16:40,685 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 676646080, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:40,685 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:40,685 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:40,686 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=6A76302C3BFA4EC0ABB0204D19EB3F8A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:40,685 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 676646080, type: INPUT, componentVertexName: F63C3F458E2B423A9F0C681C9129FA12
2016-01-14 17:16:40,687 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F63C3F458E2B423A9F0C681C9129FA12
2016-01-14 17:16:40,687 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:40,688 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 6A76302C3BFA4EC0ABB0204D19EB3F8A
2016-01-14 17:16:40,688 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:40,688 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 6A76302C3BFA4EC0ABB0204D19EB3F8A
2016-01-14 17:16:40,693 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D29E7753543149C8A034F79792BF05F2, ordinal: 1
2016-01-14 17:16:40,693 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D29E7753543149C8A034F79792BF05F2, mem on start (mb), free: 206, total: 717, max: 717
2016-01-14 17:16:40,693 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:40,694 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:40,694 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:40,696 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:40,696 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 6A76302C3BFA4EC0ABB0204D19EB3F8A
2016-01-14 17:16:40,696 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:40,696 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:40,696 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:40,697 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:40,697 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:40,697 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:40,697 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 676646080, TotalRequestedScaled: 6.245963815384616E8, TotalJVMHeap: 751828992, TotalAvailable: 503725424, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:40,697 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F63C3F458E2B423A9F0C681C9129FA12, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=676646080, allocated=503725424
2016-01-14 17:16:40,697 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 6A76302C3BFA4EC0ABB0204D19EB3F8A, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:40,698 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:40,698 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F63C3F458E2B423A9F0C681C9129FA12
2016-01-14 17:16:40,698 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:40,699 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:40,699 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: F63C3F458E2B423A9F0C681C9129FA12 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:40,699 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=676646080, postMergeMem=0, RuntimeTotalAvailable=503725424. Updated to: ShuffleMem=503725424, postMergeMem: 0
2016-01-14 17:16:40,699 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=503725424, maxSingleShuffleLimit=125931360, mergeThreshold=453352896, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:40,699 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:40,699 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F63C3F458E2B423A9F0C681C9129FA12 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:40,700 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F63C3F458E2B423A9F0C681C9129FA12
2016-01-14 17:16:40,700 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:40,701 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:40,701 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:40,701 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791800440_0001_1_01_000000_0
2016-01-14 17:16:40,702 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791800440_0001_1_00_000000_0_10200, runDuration: 39444, ]
2016-01-14 17:16:40,703 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:40,704 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: B04DE700A9044A75B75F2D3CA7269B64
2016-01-14 17:16:40,704 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_SWAP"], id: 6A76302C3BFA4EC0ABB0204D19EB3F8A
2016-01-14 17:16:40,704 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_SWAP"]
2016-01-14 17:16:40,704 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] B04DE700A9044A75B75F2D3CA7269B64, for 1 inputs
2016-01-14 17:16:40,705 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791800440_0001_1_00_000000_0_10200] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791800705, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 17:16:40,705 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : F63C3F458E2B423A9F0C681C9129FA12
2016-01-14 17:16:40,705 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:40,706 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [F63C3F458E2B423A9F0C681C9129FA12] #34 in 2ms
2016-01-14 17:16:40,706 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [F63C3F458E2B423A9F0C681C9129FA12] #34, status:false, isInterrupted:false
2016-01-14 17:16:40,706 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:40,706 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:40,707 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:40,707 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:40,707 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:40,707 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : F63C3F458E2B423A9F0C681C9129FA12
2016-01-14 17:16:40,707 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:40,707 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D29E7753543149C8A034F79792BF05F2, all 1 inputs ready in: 00:00:00.002
2016-01-14 17:16:40,708 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@3fd969ef; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@5ab9eaa8
2016-01-14 17:16:40,708 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:40,709 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(92)) - caught throwable
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:40,709 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:40,709 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:40,709 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791800440_0001_1_01_000000_0
2016-01-14 17:16:40,710 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791800440_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:40,710 INFO  task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1452791800440_0001_1_01_000000_0
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:40,711 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751828992, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791800707, Process_Duration=3, Process_End_Time=1452791800710, Read_Duration=0, Tuples_Read=1]]
2016-01-14 17:16:40,711 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D29E7753543149C8A034F79792BF05F2, mem on close (mb), free: 203, total: 717, max: 717
2016-01-14 17:16:40,711 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=D29E7753543149C8A034F79792BF05F2, index=1
2016-01-14 17:16:40,711 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791800440_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D29E7753543149C8A034F79792BF05F2, taskAttemptId=attempt_1452791800440_0001_1_01_000000_0, startTime=1452791800680, finishTime=1452791800711, timeTaken=31, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751828992, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791800707, Process_Duration=3, Process_End_Time=1452791800710, Read_Duration=0, Tuples_Read=1
2016-01-14 17:16:40,711 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:40,711 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791800440_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-01-14 17:16:40,712 INFO  impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1452791800440_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-01-14 17:16:40,712 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791800440_0001_1][Event:TASK_FINISHED]: vertexName=D29E7753543149C8A034F79792BF05F2, taskId=task_1452791800440_0001_1_01_000000, startTime=1452791800680, finishTime=1452791800712, timeTaken=32, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
], counters=Counters: 0
2016-01-14 17:16:40,712 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:40,711 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: F63C3F458E2B423A9F0C681C9129FA12
2016-01-14 17:16:40,712 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791800440_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-01-14 17:16:40,713 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:40,713 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791800440_0001_1_01 [D29E7753543149C8A034F79792BF05F2] : 1
2016-01-14 17:16:40,713 INFO  impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1452791800440_0001_1_01 [D29E7753543149C8A034F79792BF05F2] because task failed: task_1452791800440_0001_1_01_000000
2016-01-14 17:16:40,713 INFO  impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1452791800440_0001_1_01 [D29E7753543149C8A034F79792BF05F2] due to trigger: OWN_TASK_FAILURE
2016-01-14 17:16:40,713 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-01-14 17:16:40,713 INFO  impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1452791800440_0001_1_01 [D29E7753543149C8A034F79792BF05F2]
2016-01-14 17:16:40,713 INFO  impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=6A76302C3BFA4EC0ABB0204D19EB3F8A, vertexId=vertex_1452791800440_0001_1_01 [D29E7753543149C8A034F79792BF05F2]
2016-01-14 17:16:40,715 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791800440_0001_1][Event:VERTEX_FINISHED]: vertexName=D29E7753543149C8A034F79792BF05F2, vertexId=vertex_1452791800440_0001_1_01, initRequestedTime=1452791800603, initedTime=1452791800610, startRequestedTime=1452791800615, startedTime=1452791800615, finishTime=1452791800714, timeTaken=99, status=FAILED, diagnostics=Task failed, taskId=task_1452791800440_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1452791800440_0001_1_01 [D29E7753543149C8A034F79792BF05F2] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1452791800680, firstTasksToStart=[ task_1452791800440_0001_1_01_000000 ], lastTaskFinishTime=1452791800711, lastTasksToFinish=[ task_1452791800440_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[  ], longestDurationTasks=[  ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-01-14 17:16:40,715 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791800440_0001_1_01 [D29E7753543149C8A034F79792BF05F2] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-01-14 17:16:40,715 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791800440_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791800440_0001_00_000002, stoppedTime=1452791800715, exitStatus=0
2016-01-14 17:16:40,715 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791800440_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:40,715 INFO  node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1452791800440_0001_1_01_000000_0 failed: true container: container_1452791800440_0001_00_000002 numFailedTAs: 0
2016-01-14 17:16:40,715 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791800440_0001_1_01 [D29E7753543149C8A034F79792BF05F2] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-01-14 17:16:40,715 INFO  impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-01-14 17:16:40,715 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791800440_0001_1
2016-01-14 17:16:40,716 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1452791800440_0001_1_00 [F63C3F458E2B423A9F0C681C9129FA12]
2016-01-14 17:16:40,716 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 6A76302C3BFA4EC0ABB0204D19EB3F8A for vertex: vertex_1452791800440_0001_1_01 [D29E7753543149C8A034F79792BF05F2]
2016-01-14 17:16:40,716 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:40,716 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:40,719 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:40,719 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:40,719 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:40,720 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:40,720 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:40,720 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=D29E7753543149C8A034F79792BF05F2, sourceVertex=F63C3F458E2B423A9F0C681C9129FA12
2016-01-14 17:16:40,720 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=D29E7753543149C8A034F79792BF05F2, sourceVertex=6A76302C3BFA4EC0ABB0204D19EB3F8A
2016-01-14 17:16:40,720 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791800440_0001_00_000002
2016-01-14 17:16:40,720 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791800440_0001_00_000002
2016-01-14 17:16:40,720 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791800440_0001_1, queueSize=0
2016-01-14 17:16:40,720 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791800440_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-01-14 17:16:40,721 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1452791800440_0001_00_000002 completed but with errors
2016-01-14 17:16:40,721 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:40,722 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791800440_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791800440_0001_1, startTime=1452791800602, finishTime=1452791800716, timeTaken=114, status=FAILED, diagnostics=Vertex failed, vertexName=D29E7753543149C8A034F79792BF05F2, vertexId=vertex_1452791800440_0001_1_01, diagnostics=[Task failed, taskId=task_1452791800440_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1452791800440_0001_1_01 [D29E7753543149C8A034F79792BF05F2] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751828992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791800659, Process_Duration=2, Process_End_Time=1452791800661, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:40,723 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791800440_0001_1 finished with state: FAILED
2016-01-14 17:16:40,724 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791800440_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:40,724 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452791800440_0001_00_000002
2016-01-14 17:16:40,724 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1452791800440_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-01-14 17:16:40,724 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791800440_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:16:40 Completed Dag: dag_1452791800440_0001_1
2016-01-14 17:16:40,724 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791800440_0001_1, dagState=FAILED
2016-01-14 17:16:40,725 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791800440_0001_1_post
2016-01-14 17:16:40,725 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:40,725 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:40,725 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791800440_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-01-14 17:16:40,728 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...blyplatform/0_VALUES_SWAP, applicationId=application_1452791800440_0001
2016-01-14 17:16:40,729 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:40,729 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:40,729 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:40,730 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:40,730 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:40,730 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:40,730 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:40,730 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:40,730 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:40,731 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:40,731 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:40,731 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:40,731 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:40,736 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/D489E4664F28487E8AEA666FA545563B/C49AE1AC88EF4C57BA25A5A085556181/.tez/application_1452791800440_0001
2016-01-14 17:16:40,736 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping all jobs
2016-01-14 17:16:40,736 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping: (1/1) ...blyplatform/0_VALUES_SWAP
2016-01-14 17:16:40,736 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopped all jobs
2016-01-14 17:16:40,737 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.316
2016-01-14 17:16:40,737 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_VALUES_SWAP
2016-01-14 17:16:40,756 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:40,757 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:40,789 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-01-14 17:16:40,793 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.016
2016-01-14 17:16:40,793 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:40,793 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:40,793 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:40,806 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:40,811 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_ARGS_-1
2016-01-14 17:16:40,867 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:40,867 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:40,868 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_ARGS_-1"]
2016-01-14 17:16:40,868 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:40,868 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:40,868 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:40,869 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...ssemblyplatform/0_ARGS_-1
2016-01-14 17:16:40,869 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:40,880 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:40,884 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:40,885 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:40,885 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:40,894 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/4324C45E45674464A15282BAF1B63221/FDA688C5FAFE4CCD9734CE87F410A066/.tez/application_1452791800885_0001 doesn't exist and is created
2016-01-14 17:16:40,908 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:40,909 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:40,909 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/4324C45E45674464A15282BAF1B63221/FDA688C5FAFE4CCD9734CE87F410A066/.tez/application_1452791800885_0001
2016-01-14 17:16:40,912 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791800885_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:40,921 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:40,922 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:40,923 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:40,923 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:40,923 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/4324C45E45674464A15282BAF1B63221/FDA688C5FAFE4CCD9734CE87F410A066/.tez/application_1452791800885_0001/localmode-log-dir
2016-01-14 17:16:40,923 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/4324C45E45674464A15282BAF1B63221/FDA688C5FAFE4CCD9734CE87F410A066/.tez/application_1452791800885_0001/localmode-log-dir/history.txt.appattempt_1452791800885_0001_000000, maxErrors=10
2016-01-14 17:16:40,923 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:40,924 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791800885_0001_000000, appSubmitTime=1452791800911, launchTime=1452791800911
2016-01-14 17:16:40,926 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:40,927 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:40,928 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:40,928 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:40,930 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:43609
2016-01-14 17:16:40,931 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:40,933 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791800885_0001_000000, startTime=1452791800933
2016-01-14 17:16:40,933 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:40,933 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:41,009 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:41,009 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:41,010 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...ssemblyplatform/0_ARGS_-1, applicationId=application_1452791800885_0001, dagName=[4324C45E45674464A15282BAF1B63221/FDA688C5FAFE4CCD9734CE87F410A066] every/(1/1) ...ssemblyplatform/0_ARGS_-1
2016-01-14 17:16:41,024 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [4324C45E45674464A15282BAF1B63221/FDA688C5FAFE4CCD9734CE87F410A066] every/(1/1) ...ssemblyplatform/0_ARGS_-1
2016-01-14 17:16:41,025 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791800885_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/4324C45E45674464A15282BAF1B63221/FDA688C5FAFE4CCD9734CE87F410A066/.tez/application_1452791800885_0001/localmode-log-dir/dag_1452791800885_0001_1.dot
2016-01-14 17:16:41,025 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/4324C45E45674464A15282BAF1B63221/FDA688C5FAFE4CCD9734CE87F410A066/.tez/application_1452791800885_0001/localmode-log-dir/dag_1452791800885_0001_1-tez-dag.pb.txt
2016-01-14 17:16:41,031 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791800885_0001_1
2016-01-14 17:16:41,031 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:41,031 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [4324C45E45674464A15282BAF1B63221/FDA688C5FAFE4CCD9734CE87F410A066] every/(1/1) ...ssemblyplatform/0_ARGS_-1
2016-01-14 17:16:41 Running Dag: dag_1452791800885_0001_1
2016-01-14 17:16:41,041 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 17:16:41,042 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 17:16:41,042 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:41,042 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791800885_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791800885_0001_1, submitTime=1452791801024
2016-01-14 17:16:41,043 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex FBFC3410533F4679B5F4576C7A50F918
2016-01-14 17:16:41,043 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:41,043 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:41,044 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791800885_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791800885_0001_1, initTime=1452791801043
2016-01-14 17:16:41,044 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791800885_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:41,044 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791800885_0001_1][Event:DAG_STARTED]: dagID=dag_1452791800885_0001_1, startTime=1452791801044
2016-01-14 17:16:41,044 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:41,044 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791800885_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:41,044 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C3119018991143678735621D6DE35B75 : {B796560D2DAC432D804F6DC8874900D2={InputName=B796560D2DAC432D804F6DC8874900D2}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:41,044 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: B796560D2DAC432D804F6DC8874900D2, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:41,044 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791800885_0001_1_00 [C3119018991143678735621D6DE35B75]
2016-01-14 17:16:41,045 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791800885_0001_1_00 [C3119018991143678735621D6DE35B75]
2016-01-14 17:16:41,045 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791800885_0001_1_00 [C3119018991143678735621D6DE35B75]
2016-01-14 17:16:41,045 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791800885_0001_1_00 [C3119018991143678735621D6DE35B75]. Starting root input initializers: 1
2016-01-14 17:16:41,045 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:41,045 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791800885_0001_1_00 [C3119018991143678735621D6DE35B75] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:41,045 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791800885_0001_1_01 [FBFC3410533F4679B5F4576C7A50F918]
2016-01-14 17:16:41,046 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: B796560D2DAC432D804F6DC8874900D2 on vertex vertex_1452791800885_0001_1_00 [C3119018991143678735621D6DE35B75]
2016-01-14 17:16:41,046 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...ssemblyplatform/0_ARGS_-1, applicationId=application_1452791800885_0001, dagName=[4324C45E45674464A15282BAF1B63221/FDA688C5FAFE4CCD9734CE87F410A066] every/(1/1) ...ssemblyplatform/0_ARGS_-1
2016-01-14 17:16:41,048 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:41,048 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791800885_0001_1_01 [FBFC3410533F4679B5F4576C7A50F918]
2016-01-14 17:16:41,048 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:41,048 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791800885_0001_1_01 [FBFC3410533F4679B5F4576C7A50F918]
2016-01-14 17:16:41,048 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791800885_0001_1_01 [FBFC3410533F4679B5F4576C7A50F918]
2016-01-14 17:16:41,048 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=E7198A8BFBCA416C932FDFC8377C1B88, vertexId=vertex_1452791800885_0001_1_01 [FBFC3410533F4679B5F4576C7A50F918], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:41,048 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input B796560D2DAC432D804F6DC8874900D2 asking for 1 tasks. Headroom: 707 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:41,049 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=E7198A8BFBCA416C932FDFC8377C1B88, vertexId=vertex_1452791800885_0001_1_01 [FBFC3410533F4679B5F4576C7A50F918]
2016-01-14 17:16:41,049 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:41,049 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:41,050 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:41,050 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=E7198A8BFBCA416C932FDFC8377C1B88, vertexName=FBFC3410533F4679B5F4576C7A50F918, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:41,051 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: B796560D2DAC432D804F6DC8874900D2 on vertex vertex_1452791800885_0001_1_00 [C3119018991143678735621D6DE35B75]
2016-01-14 17:16:41,051 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=E7198A8BFBCA416C932FDFC8377C1B88, vertexId=vertex_1452791800885_0001_1_01 [FBFC3410533F4679B5F4576C7A50F918]
2016-01-14 17:16:41,052 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:41,052 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791800885_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FBFC3410533F4679B5F4576C7A50F918, vertexId=vertex_1452791800885_0001_1_01, initRequestedTime=1452791801045, initedTime=1452791801052, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:41,052 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791800885_0001_1_01 [FBFC3410533F4679B5F4576C7A50F918] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:41,053 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791800885_0001_1_00 [C3119018991143678735621D6DE35B75] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:41,053 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {B796560D2DAC432D804F6DC8874900D2=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:41,053 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791800885_0001_1_00 [C3119018991143678735621D6DE35B75] parallelism set to 1
2016-01-14 17:16:41,053 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791800885_0001_1_00 [C3119018991143678735621D6DE35B75] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:41,054 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791800885_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791800885_0001_1_00, eventCount=1
2016-01-14 17:16:41,054 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791800885_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C3119018991143678735621D6DE35B75, vertexId=vertex_1452791800885_0001_1_00, initRequestedTime=1452791801044, initedTime=1452791801054, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:41,054 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:41,054 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791800885_0001_1_00 [C3119018991143678735621D6DE35B75] with distanceFromRoot: 0
2016-01-14 17:16:41,055 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791800885_0001_1_00 [C3119018991143678735621D6DE35B75] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:41,055 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C3119018991143678735621D6DE35B75
2016-01-14 17:16:41,055 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791800885_0001_1_00 [C3119018991143678735621D6DE35B75]
2016-01-14 17:16:41,055 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:41,056 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791800885_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791800885_0001_1_00, startRequestedTime=1452791801055, startedTime=1452791801055
2016-01-14 17:16:41,056 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791800885_0001_1_00 [C3119018991143678735621D6DE35B75] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:41,056 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:41,056 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791800885_0001_1][Event:TASK_STARTED]: vertexName=C3119018991143678735621D6DE35B75, taskId=task_1452791800885_0001_1_00_000000, scheduledTime=1452791801056, launchTime=1452791801056
2016-01-14 17:16:41,057 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791800885_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:41,057 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791800885_0001_1_00 for vertex: vertex_1452791800885_0001_1_01 [FBFC3410533F4679B5F4576C7A50F918] numStartedSources: 1 numSources: 1
2016-01-14 17:16:41,057 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791800885_0001_1_01 [FBFC3410533F4679B5F4576C7A50F918] with distanceFromRoot: 1
2016-01-14 17:16:41,057 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791800885_0001_1_00 [C3119018991143678735621D6DE35B75]
2016-01-14 17:16:41,057 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: C3119018991143678735621D6DE35B75 in vertex: FBFC3410533F4679B5F4576C7A50F918
2016-01-14 17:16:41,058 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: FBFC3410533F4679B5F4576C7A50F918 with 1 source tasks and 1 pending tasks
2016-01-14 17:16:41,058 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:41,058 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791800885_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791800885_0001_1_01, startRequestedTime=1452791801057, startedTime=1452791801057
2016-01-14 17:16:41,058 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791800885_0001_1_01 [FBFC3410533F4679B5F4576C7A50F918] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:41,058 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [4324C45E45674464A15282BAF1B63221/FDA688C5FAFE4CCD9734CE87F410A066] every/(1/1) ...ssemblyplatform/0_ARGS_-1, VertexName: C3119018991143678735621D6DE35B75, VertexParallelism: 1, TaskAttemptID:attempt_1452791800885_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B796560D2DAC432D804F6DC8874900D2, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FBFC3410533F4679B5F4576C7A50F918, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:41,059 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791800885_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:41,059 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:41,059 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:41,060 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:41,060 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:41,060 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791800885_0001_00_000001
2016-01-14 17:16:41,060 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791800885_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:41,062 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791800885_0001_00_000001
2016-01-14 17:16:41,062 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:41,062 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791800885_0001, with dag id: dag_1452791800885_0001_1
2016-01-14 17:16:41,063 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791800885_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:41,063 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791800885_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791800885_0001_00_000001, launchTime=1452791801063
2016-01-14 17:16:41,063 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:41,064 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791800885_0001_00_000001
2016-01-14 17:16:41,065 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791800885_0001_1_00_000000_0] to container: [container_1452791800885_0001_00_000001]
2016-01-14 17:16:41,065 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791800885_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:41,066 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791800885_0001_1_00_000000_0] started. Is using containerId: [container_1452791800885_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:41,066 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791800885_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C3119018991143678735621D6DE35B75, taskAttemptId=attempt_1452791800885_0001_1_00_000000_0, startTime=1452791801065, containerId=container_1452791800885_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791800885_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:41,066 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:41,066 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791800885_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:41,066 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791800885_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:41,066 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791800885_0001_00_000001 given task: attempt_1452791800885_0001_1_00_000000_0
2016-01-14 17:16:41,066 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791800885_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791800885_0001_1_00_000000_0
2016-01-14 17:16:41 Starting to run new task attempt: attempt_1452791800885_0001_1_00_000000_0
2016-01-14 17:16:41,067 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791800885_0001_1_00_000000_0
2016-01-14 17:16:41,067 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:41,068 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:41,068 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:41,068 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:41,068 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:41,068 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:41,068 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [4324C45E45674464A15282BAF1B63221/FDA688C5FAFE4CCD9734CE87F410A066] every/(1/1) ...ssemblyplatform/0_ARGS_-1, VertexName: C3119018991143678735621D6DE35B75, VertexParallelism: 1, TaskAttemptID:attempt_1452791800885_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B796560D2DAC432D804F6DC8874900D2, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FBFC3410533F4679B5F4576C7A50F918, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:41,069 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=741867520
2016-01-14 17:16:41,069 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791800885_0001_1_00_000000_0
2016-01-14 17:16:41,069 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:41,069 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:41,070 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791800885_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:41,070 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:41,070 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B796560D2DAC432D804F6DC8874900D2, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:41,070 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=FBFC3410533F4679B5F4576C7A50F918, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:41,070 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:41,071 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B796560D2DAC432D804F6DC8874900D2
2016-01-14 17:16:41,071 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: B796560D2DAC432D804F6DC8874900D2
2016-01-14 17:16:41,071 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:41,071 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: FBFC3410533F4679B5F4576C7A50F918
2016-01-14 17:16:41,073 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:41,073 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: FBFC3410533F4679B5F4576C7A50F918
2016-01-14 17:16:41,073 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: FBFC3410533F4679B5F4576C7A50F918
2016-01-14 17:16:41,073 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:41,073 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: B796560D2DAC432D804F6DC8874900D2
2016-01-14 17:16:41,073 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B796560D2DAC432D804F6DC8874900D2
2016-01-14 17:16:41,074 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:41,077 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C3119018991143678735621D6DE35B75, ordinal: 0
2016-01-14 17:16:41,077 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C3119018991143678735621D6DE35B75, mem on start (mb), free: 313, total: 707, max: 707
2016-01-14 17:16:41,078 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:41,078 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:41,078 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:41,078 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:41,079 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:41,079 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:41,079 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:41,079 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:41,079 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:41,080 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 741867520, TotalAvailable: 497051238, TotalRequested/TotalJVMHeap:0.14
2016-01-14 17:16:41,080 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B796560D2DAC432D804F6DC8874900D2, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:41,080 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, FBFC3410533F4679B5F4576C7A50F918, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:41,081 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:41,081 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B796560D2DAC432D804F6DC8874900D2 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:41,081 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:41,081 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:41,081 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B796560D2DAC432D804F6DC8874900D2
2016-01-14 17:16:41,082 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:41,082 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B796560D2DAC432D804F6DC8874900D2
2016-01-14 17:16:41,082 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:41,083 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791800885_0001_1_00_000000_0
2016-01-14 17:16:41,085 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:41,085 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: B796560D2DAC432D804F6DC8874900D2
2016-01-14 17:16:41,085 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 1E091E1F07D64E3FA0142B28453680DB
2016-01-14 17:16:41,086 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 1E091E1F07D64E3FA0142B28453680DB
2016-01-14 17:16:41,086 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:41,086 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@480ee892; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@4ac6ee33; comparator=cascading.tuple.hadoop.util.TupleComparator@4a407c9f; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:41,086 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:41,086 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@4fb5d0ce
2016-01-14 17:16:41,086 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:41,087 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:41,089 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@4fb5d0ce
2016-01-14 17:16:41,089 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:41,089 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:41,100 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:41,100 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:41,100 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:41,100 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:41,100 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:41,101 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:41,101 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:41,101 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C3119018991143678735621D6DE35B75, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:41,104 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791800885_0001_1_00_000000_0
2016-01-14 17:16:41,105 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C3119018991143678735621D6DE35B75, mem on close (mb), free: 211, total: 707, max: 707
2016-01-14 17:16:41,105 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:41,105 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:41,105 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:41,105 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:41,108 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:41 Completed running task attempt: attempt_1452791800885_0001_1_00_000000_0
2016-01-14 17:16:41,111 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791800885_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:41,111 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791800885_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791800885_0001_1_00, eventCount=1
2016-01-14 17:16:41,111 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=741867520, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791801101, Process_Duration=3, Process_End_Time=1452791801104, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:41,111 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:41,112 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791800885_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C3119018991143678735621D6DE35B75, taskAttemptId=attempt_1452791800885_0001_1_00_000000_0, startTime=1452791801065, finishTime=1452791801111, timeTaken=46, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=741867520, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791801101, Process_Duration=3, Process_End_Time=1452791801104, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:41,112 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:41,112 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791800885_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:41,112 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791800885_0001_1_00_000000_0
2016-01-14 17:16:41,112 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:41,112 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791800885_0001_1][Event:TASK_FINISHED]: vertexName=C3119018991143678735621D6DE35B75, taskId=task_1452791800885_0001_1_00_000000, startTime=1452791801065, finishTime=1452791801112, timeTaken=47, status=SUCCEEDED, successfulAttemptID=attempt_1452791800885_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=741867520, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791801101, Process_Duration=3, Process_End_Time=1452791801104, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:41,112 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791800885_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:41,113 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791800885_0001_1_00 [C3119018991143678735621D6DE35B75] : 1
2016-01-14 17:16:41,113 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791800885_0001_1_00 [C3119018991143678735621D6DE35B75]
2016-01-14 17:16:41,112 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:41,113 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:41,113 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:41,113 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791800885_0001_1][Event:VERTEX_FINISHED]: vertexName=C3119018991143678735621D6DE35B75, vertexId=vertex_1452791800885_0001_1_00, initRequestedTime=1452791801044, initedTime=1452791801054, startRequestedTime=1452791801055, startedTime=1452791801055, finishTime=1452791801113, timeTaken=58, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=741867520, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791801101, Process_Duration=3, Process_End_Time=1452791801104, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791801065, firstTasksToStart=[ task_1452791800885_0001_1_00_000000 ], lastTaskFinishTime=1452791801111, lastTasksToFinish=[ task_1452791800885_0001_1_00_000000 ], minTaskDuration=46, maxTaskDuration=46, avgTaskDuration=46.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791800885_0001_1_00_000000 ], longestDurationTasks=[ task_1452791800885_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:41,114 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791800885_0001_1_00 [C3119018991143678735621D6DE35B75] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:41,114 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791800885_0001_1_01 [FBFC3410533F4679B5F4576C7A50F918] attempt: attempt_1452791800885_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:41,114 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: FBFC3410533F4679B5F4576C7A50F918
2016-01-14 17:16:41,114 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791800885_0001_1_01 [FBFC3410533F4679B5F4576C7A50F918]
2016-01-14 17:16:41,114 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791800885_0001_00_000001
2016-01-14 17:16:41,114 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791800885_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:41,114 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791800885_0001_1_00 [C3119018991143678735621D6DE35B75] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:41,114 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791800885_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791800885_0001_00_000001, stoppedTime=1452791801114, exitStatus=0
2016-01-14 17:16:41,114 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:41,115 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791800885_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:41,115 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(221)) - No more tasks for container with id : container_1452791800885_0001_00_000001. Asking it to die
2016-01-14 17:16:41,115 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791800885_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:41,115 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791800885_0001_1][Event:TASK_STARTED]: vertexName=FBFC3410533F4679B5F4576C7A50F918, taskId=task_1452791800885_0001_1_01_000000, scheduledTime=1452791801115, launchTime=1452791801115
2016-01-14 17:16:41,115 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791800885_0001_00_000001, Exiting
2016-01-14 17:16:41,115 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791800885_0001_00_000001
2016-01-14 17:16:41,115 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791800885_0001_00_000001
2016-01-14 17:16:41,115 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:41,115 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791800885_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:41,116 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791800885_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:41,116 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791800885_0001_1_01 [FBFC3410533F4679B5F4576C7A50F918]
2016-01-14 17:16:41,116 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791800885_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:41,116 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [4324C45E45674464A15282BAF1B63221/FDA688C5FAFE4CCD9734CE87F410A066] every/(1/1) ...ssemblyplatform/0_ARGS_-1, VertexName: FBFC3410533F4679B5F4576C7A50F918, VertexParallelism: 1, TaskAttemptID:attempt_1452791800885_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C3119018991143678735621D6DE35B75, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=E7198A8BFBCA416C932FDFC8377C1B88, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:41,116 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791800885_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:41,117 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:41,117 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791800885_0001_00_000002
2016-01-14 17:16:41,117 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791800885_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:41,118 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791800885_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:41,118 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791800885_0001_00_000001 completed successfully
2016-01-14 17:16:41,119 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791800885_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:41,119 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791800885_0001_00_000002
2016-01-14 17:16:41,120 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 17:16:41,122 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791800885_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:41,122 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791800885_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791800885_0001_00_000002, launchTime=1452791801122
2016-01-14 17:16:41,122 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:41,123 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791800885_0001_00_000002
2016-01-14 17:16:41,123 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791800885_0001_1_01_000000_0] to container: [container_1452791800885_0001_00_000002]
2016-01-14 17:16:41,123 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791800885_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:41,123 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791800885_0001_00_000002 given task: attempt_1452791800885_0001_1_01_000000_0
2016-01-14 17:16:41,123 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791800885_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791800885_0001_1_01_000000_0
2016-01-14 17:16:41 Starting to run new task attempt: attempt_1452791800885_0001_1_01_000000_0
2016-01-14 17:16:41,124 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791800885_0001_1_01_000000_0] started. Is using containerId: [container_1452791800885_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:41,124 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791800885_0001_1_01_000000_0
2016-01-14 17:16:41,124 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:41,124 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:41,124 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:41,124 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:41,124 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791800885_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FBFC3410533F4679B5F4576C7A50F918, taskAttemptId=attempt_1452791800885_0001_1_01_000000_0, startTime=1452791801123, containerId=container_1452791800885_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791800885_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:41,124 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:41,124 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:41,124 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:41,125 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [4324C45E45674464A15282BAF1B63221/FDA688C5FAFE4CCD9734CE87F410A066] every/(1/1) ...ssemblyplatform/0_ARGS_-1, VertexName: FBFC3410533F4679B5F4576C7A50F918, VertexParallelism: 1, TaskAttemptID:attempt_1452791800885_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C3119018991143678735621D6DE35B75, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=E7198A8BFBCA416C932FDFC8377C1B88, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:41,124 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791800885_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:41,125 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=741867520
2016-01-14 17:16:41,125 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791800885_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:41,125 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791800885_0001_1_01_000000_0
2016-01-14 17:16:41,125 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:41,125 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:41,126 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:41,126 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791800885_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:41,126 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C3119018991143678735621D6DE35B75, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:41,126 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:41,126 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C3119018991143678735621D6DE35B75
2016-01-14 17:16:41,128 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 667680768, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:41,128 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:41,128 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 667680768, type: INPUT, componentVertexName: C3119018991143678735621D6DE35B75
2016-01-14 17:16:41,128 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C3119018991143678735621D6DE35B75
2016-01-14 17:16:41,128 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:41,128 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=E7198A8BFBCA416C932FDFC8377C1B88, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:41,129 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:41,129 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: E7198A8BFBCA416C932FDFC8377C1B88
2016-01-14 17:16:41,129 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:41,129 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: E7198A8BFBCA416C932FDFC8377C1B88
2016-01-14 17:16:41,135 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FBFC3410533F4679B5F4576C7A50F918, ordinal: 1
2016-01-14 17:16:41,135 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FBFC3410533F4679B5F4576C7A50F918, mem on start (mb), free: 206, total: 707, max: 707
2016-01-14 17:16:41,135 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:41,135 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:41,135 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:41,141 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:41,141 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: E7198A8BFBCA416C932FDFC8377C1B88
2016-01-14 17:16:41,141 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:41,141 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:41,141 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:41,141 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:41,142 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:41,142 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:41,142 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 667680768, TotalRequestedScaled: 6.16320708923077E8, TotalJVMHeap: 741867520, TotalAvailable: 497051238, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:41,142 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C3119018991143678735621D6DE35B75, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=667680768, allocated=497051238
2016-01-14 17:16:41,142 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, E7198A8BFBCA416C932FDFC8377C1B88, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:41,142 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:41,142 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C3119018991143678735621D6DE35B75 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:41,142 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:41,142 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:41,142 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C3119018991143678735621D6DE35B75
2016-01-14 17:16:41,143 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:41,143 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:41,144 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: C3119018991143678735621D6DE35B75 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:41,144 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=667680768, postMergeMem=0, RuntimeTotalAvailable=497051238. Updated to: ShuffleMem=497051238, postMergeMem: 0
2016-01-14 17:16:41,144 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=497051238, maxSingleShuffleLimit=124262808, mergeThreshold=447346112, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:41,144 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:41,146 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C3119018991143678735621D6DE35B75
2016-01-14 17:16:41,146 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:41,147 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791800885_0001_1_01_000000_0
2016-01-14 17:16:41,147 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791800885_0001_1_00_000000_0_10207, runDuration: 38334, ]
2016-01-14 17:16:41,148 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791800885_0001_1_00_000000_0_10207] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791801148, TimeTaken=0, Rate=0.00 MB/s
2016-01-14 17:16:41,148 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : C3119018991143678735621D6DE35B75
2016-01-14 17:16:41,149 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:41,149 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [C3119018991143678735621D6DE35B75] #35 in 1ms
2016-01-14 17:16:41,149 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [C3119018991143678735621D6DE35B75] #35, status:false, isInterrupted:false
2016-01-14 17:16:41,150 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:41,151 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 1E091E1F07D64E3FA0142B28453680DB
2016-01-14 17:16:41,151 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_ARGS_-1"], id: E7198A8BFBCA416C932FDFC8377C1B88
2016-01-14 17:16:41,151 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_ARGS_-1"]
2016-01-14 17:16:41,151 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 1E091E1F07D64E3FA0142B28453680DB, for 1 inputs
2016-01-14 17:16:41,151 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:41,152 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:41,152 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:41,152 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:41,152 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:41,152 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : C3119018991143678735621D6DE35B75
2016-01-14 17:16:41,152 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:41,152 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FBFC3410533F4679B5F4576C7A50F918, all 1 inputs ready in: 00:00:00.001
2016-01-14 17:16:41,154 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@6c807f3b; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@68213223
2016-01-14 17:16:41,158 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:41,158 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:41,159 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791800885_0001_1_01_000000_0
2016-01-14 17:16:41,159 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791800885_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:41,159 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791800885_0001_1_01_000000_0
2016-01-14 17:16:41,159 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FBFC3410533F4679B5F4576C7A50F918, mem on close (mb), free: 202, total: 707, max: 707
2016-01-14 17:16:41,159 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: C3119018991143678735621D6DE35B75
2016-01-14 17:16:41,160 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:41,160 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:41,160 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:41,160 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:41,160 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:41 Completed running task attempt: attempt_1452791800885_0001_1_01_000000_0
2016-01-14 17:16:41,161 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791800885_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:41,161 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=652, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=741867520, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791801152, Process_Duration=7, Process_End_Time=1452791801159, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 17:16:41,161 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:41,161 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791800885_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FBFC3410533F4679B5F4576C7A50F918, taskAttemptId=attempt_1452791800885_0001_1_01_000000_0, startTime=1452791801123, finishTime=1452791801161, timeTaken=38, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=652, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=741867520, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791801152, Process_Duration=7, Process_End_Time=1452791801159, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:41,161 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:41,161 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791800885_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:41,162 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:41,162 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791800885_0001_1_01_000000_0
2016-01-14 17:16:41,162 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:41,162 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791800885_0001_00_000002
2016-01-14 17:16:41,162 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791800885_0001_1][Event:TASK_FINISHED]: vertexName=FBFC3410533F4679B5F4576C7A50F918, taskId=task_1452791800885_0001_1_01_000000, startTime=1452791801123, finishTime=1452791801162, timeTaken=39, status=SUCCEEDED, successfulAttemptID=attempt_1452791800885_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=652, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=741867520, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791801152, Process_Duration=7, Process_End_Time=1452791801159, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:41,162 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791800885_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:41,162 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:41,162 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791800885_0001_1_01 [FBFC3410533F4679B5F4576C7A50F918] : 1
2016-01-14 17:16:41,163 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791800885_0001_1_01 [FBFC3410533F4679B5F4576C7A50F918]
2016-01-14 17:16:41,163 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:41,163 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791800885_0001_1][Event:VERTEX_FINISHED]: vertexName=FBFC3410533F4679B5F4576C7A50F918, vertexId=vertex_1452791800885_0001_1_01, initRequestedTime=1452791801045, initedTime=1452791801052, startRequestedTime=1452791801057, startedTime=1452791801057, finishTime=1452791801163, timeTaken=106, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=652, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=741867520, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791801152, Process_Duration=7, Process_End_Time=1452791801159, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452791801123, firstTasksToStart=[ task_1452791800885_0001_1_01_000000 ], lastTaskFinishTime=1452791801161, lastTasksToFinish=[ task_1452791800885_0001_1_01_000000 ], minTaskDuration=38, maxTaskDuration=38, avgTaskDuration=38.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791800885_0001_1_01_000000 ], longestDurationTasks=[ task_1452791800885_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:41,163 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791800885_0001_1_01 [FBFC3410533F4679B5F4576C7A50F918] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:41,163 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791800885_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:41,164 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791800885_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791800885_0001_00_000002, stoppedTime=1452791801164, exitStatus=0
2016-01-14 17:16:41,164 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791800885_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:41,164 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791800885_0001_1_01 [FBFC3410533F4679B5F4576C7A50F918] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:41,164 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:41,164 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791800885_0001_1
2016-01-14 17:16:41,164 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791800885_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452791800885_0001_1
2016-01-14 17:16:41,164 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: E7198A8BFBCA416C932FDFC8377C1B88 for vertex: vertex_1452791800885_0001_1_01 [FBFC3410533F4679B5F4576C7A50F918]
2016-01-14 17:16:41,172 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791800885_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 17:16:41,173 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791800885_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:41,173 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791800885_0001_00_000002, Exiting
2016-01-14 17:16:41,173 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791800885_0001_00_000002
2016-01-14 17:16:41,173 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791800885_0001_00_000002
2016-01-14 17:16:41,173 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791800885_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:41,173 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452791800885_0001_1_00 [C3119018991143678735621D6DE35B75]
2016-01-14 17:16:41,173 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791800885_0001_00_000002 completed successfully
2016-01-14 17:16:41,175 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791800885_0001_1, queueSize=0
2016-01-14 17:16:41,175 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:41,176 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791800885_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791800885_0001_1, startTime=1452791801044, finishTime=1452791801173, timeTaken=129, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=985, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1483735040, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905583602253, Process_Duration=10, Process_End_Time=2905583602263, Read_Duration=4, Tuples_Read=40, Tuples_Written=40, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 17:16:41,176 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791800885_0001_1 finished with state: SUCCEEDED
2016-01-14 17:16:41,176 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791800885_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:41,177 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791800885_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:16:41,177 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452791800885_0001_00_000002
2016-01-14 17:16:41 Completed Dag: dag_1452791800885_0001_1
2016-01-14 17:16:41,179 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791800885_0001_1, dagState=SUCCEEDED
2016-01-14 17:16:41,180 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791800885_0001_1_post
2016-01-14 17:16:41,181 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:41,181 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:41,182 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...ssemblyplatform/0_ARGS_-1, applicationId=application_1452791800885_0001
2016-01-14 17:16:41,183 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:41,183 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:41,183 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:41,183 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:41,184 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:41,184 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:41,184 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:41,184 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:41,185 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:41,184 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:41,185 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:41,185 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:41,185 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:41,190 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/4324C45E45674464A15282BAF1B63221/FDA688C5FAFE4CCD9734CE87F410A066/.tez/application_1452791800885_0001
2016-01-14 17:16:41,190 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.323
2016-01-14 17:16:41,257 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_ARGS_-1
2016-01-14 17:16:41,279 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:41,280 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:41,304 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.003
2016-01-14 17:16:41,308 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.006
2016-01-14 17:16:41,308 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 17:16:41,308 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 17:16:41,308 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_ARGS_'field'
2016-01-14 17:16:41,346 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:41,346 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:41,374 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.006
2016-01-14 17:16:41,379 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.011
2016-01-14 17:16:41,380 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:41,380 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:41,380 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:41,393 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:41,398 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_ARGS_RESULTS
2016-01-14 17:16:41,437 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:41,438 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:41,438 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_ARGS_RESULTS"]
2016-01-14 17:16:41,438 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:41,438 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:41,438 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:41,439 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...lyplatform/0_ARGS_RESULTS
2016-01-14 17:16:41,439 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:41,450 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:41,456 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:41,456 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:41,456 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:41,466 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/45B5FB3103C7450FB8A875B4FFE73693/C3EAE9C5835D4952BEBACEF814BE8695/.tez/application_1452791801456_0001 doesn't exist and is created
2016-01-14 17:16:41,481 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:41,481 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:41,481 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/45B5FB3103C7450FB8A875B4FFE73693/C3EAE9C5835D4952BEBACEF814BE8695/.tez/application_1452791801456_0001
2016-01-14 17:16:41,484 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791801456_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:41,494 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:41,495 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:41,497 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:41,497 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:41,497 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/45B5FB3103C7450FB8A875B4FFE73693/C3EAE9C5835D4952BEBACEF814BE8695/.tez/application_1452791801456_0001/localmode-log-dir
2016-01-14 17:16:41,498 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/45B5FB3103C7450FB8A875B4FFE73693/C3EAE9C5835D4952BEBACEF814BE8695/.tez/application_1452791801456_0001/localmode-log-dir/history.txt.appattempt_1452791801456_0001_000000, maxErrors=10
2016-01-14 17:16:41,498 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:41,499 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791801456_0001_000000, appSubmitTime=1452791801484, launchTime=1452791801484
2016-01-14 17:16:41,503 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:53997
2016-01-14 17:16:41,507 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:41,507 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:41,508 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:41,508 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:41,509 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:41,511 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791801456_0001_000000, startTime=1452791801511
2016-01-14 17:16:41,511 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:41,511 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:41,581 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:41,581 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:41,581 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...lyplatform/0_ARGS_RESULTS, applicationId=application_1452791801456_0001, dagName=[45B5FB3103C7450FB8A875B4FFE73693/C3EAE9C5835D4952BEBACEF814BE8695] every/(1/1) ...lyplatform/0_ARGS_RESULTS
2016-01-14 17:16:41,597 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [45B5FB3103C7450FB8A875B4FFE73693/C3EAE9C5835D4952BEBACEF814BE8695] every/(1/1) ...lyplatform/0_ARGS_RESULTS
2016-01-14 17:16:41,598 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791801456_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/45B5FB3103C7450FB8A875B4FFE73693/C3EAE9C5835D4952BEBACEF814BE8695/.tez/application_1452791801456_0001/localmode-log-dir/dag_1452791801456_0001_1.dot
2016-01-14 17:16:41,599 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/45B5FB3103C7450FB8A875B4FFE73693/C3EAE9C5835D4952BEBACEF814BE8695/.tez/application_1452791801456_0001/localmode-log-dir/dag_1452791801456_0001_1-tez-dag.pb.txt
2016-01-14 17:16:41,605 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791801456_0001_1
2016-01-14 17:16:41,605 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:41,605 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [45B5FB3103C7450FB8A875B4FFE73693/C3EAE9C5835D4952BEBACEF814BE8695] every/(1/1) ...lyplatform/0_ARGS_RESULTS
2016-01-14 17:16:41 Running Dag: dag_1452791801456_0001_1
2016-01-14 17:16:41,617 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791801456_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791801456_0001_1, submitTime=1452791801597
2016-01-14 17:16:41,617 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:41,618 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex F1C5B015C6614631B4F092D734B8406D
2016-01-14 17:16:41,618 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:41,618 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791801456_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791801456_0001_1, initTime=1452791801618
2016-01-14 17:16:41,618 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:41,619 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791801456_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:41,619 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791801456_0001_1][Event:DAG_STARTED]: dagID=dag_1452791801456_0001_1, startTime=1452791801619
2016-01-14 17:16:41,619 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:41,619 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791801456_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:41,619 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: F7D408974AA14669B3E4B14DFE215782 : {516C6AD6891D40139033401324E863D8={InputName=516C6AD6891D40139033401324E863D8}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:41,619 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 516C6AD6891D40139033401324E863D8, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:41,619 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791801456_0001_1_00 [F7D408974AA14669B3E4B14DFE215782]
2016-01-14 17:16:41,619 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791801456_0001_1_00 [F7D408974AA14669B3E4B14DFE215782]
2016-01-14 17:16:41,620 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791801456_0001_1_00 [F7D408974AA14669B3E4B14DFE215782]
2016-01-14 17:16:41,620 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:41,620 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791801456_0001_1_00 [F7D408974AA14669B3E4B14DFE215782]. Starting root input initializers: 1
2016-01-14 17:16:41,620 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791801456_0001_1_00 [F7D408974AA14669B3E4B14DFE215782] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:41,620 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791801456_0001_1_01 [F1C5B015C6614631B4F092D734B8406D]
2016-01-14 17:16:41,621 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...lyplatform/0_ARGS_RESULTS, applicationId=application_1452791801456_0001, dagName=[45B5FB3103C7450FB8A875B4FFE73693/C3EAE9C5835D4952BEBACEF814BE8695] every/(1/1) ...lyplatform/0_ARGS_RESULTS
2016-01-14 17:16:41,621 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 516C6AD6891D40139033401324E863D8 on vertex vertex_1452791801456_0001_1_00 [F7D408974AA14669B3E4B14DFE215782]
2016-01-14 17:16:41,623 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:41,623 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 516C6AD6891D40139033401324E863D8 asking for 1 tasks. Headroom: 717 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:41,623 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:41,623 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791801456_0001_1_01 [F1C5B015C6614631B4F092D734B8406D]
2016-01-14 17:16:41,623 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:41,624 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791801456_0001_1_01 [F1C5B015C6614631B4F092D734B8406D]
2016-01-14 17:16:41,624 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791801456_0001_1_01 [F1C5B015C6614631B4F092D734B8406D]
2016-01-14 17:16:41,625 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=03E55A9D901746C9B9225F912368B897, vertexId=vertex_1452791801456_0001_1_01 [F1C5B015C6614631B4F092D734B8406D], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:41,625 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:41,625 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=03E55A9D901746C9B9225F912368B897, vertexId=vertex_1452791801456_0001_1_01 [F1C5B015C6614631B4F092D734B8406D]
2016-01-14 17:16:41,626 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:41,626 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 516C6AD6891D40139033401324E863D8 on vertex vertex_1452791801456_0001_1_00 [F7D408974AA14669B3E4B14DFE215782]
2016-01-14 17:16:41,627 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=03E55A9D901746C9B9225F912368B897, vertexName=F1C5B015C6614631B4F092D734B8406D, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:41,627 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=03E55A9D901746C9B9225F912368B897, vertexId=vertex_1452791801456_0001_1_01 [F1C5B015C6614631B4F092D734B8406D]
2016-01-14 17:16:41,628 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:41,629 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791801456_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F1C5B015C6614631B4F092D734B8406D, vertexId=vertex_1452791801456_0001_1_01, initRequestedTime=1452791801620, initedTime=1452791801628, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:41,629 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791801456_0001_1_01 [F1C5B015C6614631B4F092D734B8406D] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:41,629 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791801456_0001_1_00 [F7D408974AA14669B3E4B14DFE215782] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:41,629 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {516C6AD6891D40139033401324E863D8=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:41,630 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791801456_0001_1_00 [F7D408974AA14669B3E4B14DFE215782] parallelism set to 1
2016-01-14 17:16:41,630 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791801456_0001_1_00 [F7D408974AA14669B3E4B14DFE215782] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:41,630 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791801456_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791801456_0001_1_00, eventCount=1
2016-01-14 17:16:41,630 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:41,631 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791801456_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F7D408974AA14669B3E4B14DFE215782, vertexId=vertex_1452791801456_0001_1_00, initRequestedTime=1452791801619, initedTime=1452791801630, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:41,631 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791801456_0001_1_00 [F7D408974AA14669B3E4B14DFE215782] with distanceFromRoot: 0
2016-01-14 17:16:41,631 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791801456_0001_1_00 [F7D408974AA14669B3E4B14DFE215782] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:41,631 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in F7D408974AA14669B3E4B14DFE215782
2016-01-14 17:16:41,631 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791801456_0001_1_00 [F7D408974AA14669B3E4B14DFE215782]
2016-01-14 17:16:41,632 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:41,632 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791801456_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791801456_0001_1_00, startRequestedTime=1452791801631, startedTime=1452791801631
2016-01-14 17:16:41,632 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791801456_0001_1_00 [F7D408974AA14669B3E4B14DFE215782] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:41,633 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:41,633 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791801456_0001_1][Event:TASK_STARTED]: vertexName=F7D408974AA14669B3E4B14DFE215782, taskId=task_1452791801456_0001_1_00_000000, scheduledTime=1452791801632, launchTime=1452791801632
2016-01-14 17:16:41,633 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791801456_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:41,633 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791801456_0001_1_00 for vertex: vertex_1452791801456_0001_1_01 [F1C5B015C6614631B4F092D734B8406D] numStartedSources: 1 numSources: 1
2016-01-14 17:16:41,633 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791801456_0001_1_01 [F1C5B015C6614631B4F092D734B8406D] with distanceFromRoot: 1
2016-01-14 17:16:41,634 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791801456_0001_1_00 [F7D408974AA14669B3E4B14DFE215782]
2016-01-14 17:16:41,634 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: F7D408974AA14669B3E4B14DFE215782 in vertex: F1C5B015C6614631B4F092D734B8406D
2016-01-14 17:16:41,634 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: F1C5B015C6614631B4F092D734B8406D with 1 source tasks and 1 pending tasks
2016-01-14 17:16:41,635 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:41,635 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791801456_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791801456_0001_1_01, startRequestedTime=1452791801634, startedTime=1452791801634
2016-01-14 17:16:41,635 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791801456_0001_1_01 [F1C5B015C6614631B4F092D734B8406D] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:41,635 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [45B5FB3103C7450FB8A875B4FFE73693/C3EAE9C5835D4952BEBACEF814BE8695] every/(1/1) ...lyplatform/0_ARGS_RESULTS, VertexName: F7D408974AA14669B3E4B14DFE215782, VertexParallelism: 1, TaskAttemptID:attempt_1452791801456_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=516C6AD6891D40139033401324E863D8, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F1C5B015C6614631B4F092D734B8406D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:41,637 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791801456_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:41,636 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791801456_0001, with dag id: dag_1452791801456_0001_1
2016-01-14 17:16:41,638 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:41,639 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:41,639 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:41,639 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:41,640 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791801456_0001_00_000001
2016-01-14 17:16:41,640 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 17:16:41,640 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 17:16:41,640 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791801456_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:41,641 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791801456_0001_00_000001
2016-01-14 17:16:41,641 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:41,642 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791801456_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:41,642 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791801456_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791801456_0001_00_000001, launchTime=1452791801642
2016-01-14 17:16:41,642 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:41,643 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791801456_0001_00_000001
2016-01-14 17:16:41,643 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791801456_0001_1_00_000000_0] to container: [container_1452791801456_0001_00_000001]
2016-01-14 17:16:41,643 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791801456_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:41,644 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791801456_0001_00_000001 given task: attempt_1452791801456_0001_1_00_000000_0
2016-01-14 17:16:41,644 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791801456_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791801456_0001_1_00_000000_0
2016-01-14 17:16:41 Starting to run new task attempt: attempt_1452791801456_0001_1_00_000000_0
2016-01-14 17:16:41,644 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791801456_0001_1_00_000000_0
2016-01-14 17:16:41,644 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:41,644 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:41,644 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:41,644 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:41,644 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:41,645 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:41,645 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [45B5FB3103C7450FB8A875B4FFE73693/C3EAE9C5835D4952BEBACEF814BE8695] every/(1/1) ...lyplatform/0_ARGS_RESULTS, VertexName: F7D408974AA14669B3E4B14DFE215782, VertexParallelism: 1, TaskAttemptID:attempt_1452791801456_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=516C6AD6891D40139033401324E863D8, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F1C5B015C6614631B4F092D734B8406D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:41,645 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=751828992
2016-01-14 17:16:41,645 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791801456_0001_1_00_000000_0
2016-01-14 17:16:41,645 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:41,646 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:41,646 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791801456_0001_1_00_000000_0] started. Is using containerId: [container_1452791801456_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:41,645 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791801456_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:41,646 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:41,646 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:41,647 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=516C6AD6891D40139033401324E863D8, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:41,647 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791801456_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F7D408974AA14669B3E4B14DFE215782, taskAttemptId=attempt_1452791801456_0001_1_00_000000_0, startTime=1452791801646, containerId=container_1452791801456_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791801456_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:41,647 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791801456_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:41,647 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:41,648 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 516C6AD6891D40139033401324E863D8
2016-01-14 17:16:41,648 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 516C6AD6891D40139033401324E863D8
2016-01-14 17:16:41,648 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F1C5B015C6614631B4F092D734B8406D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:41,647 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791801456_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:41,649 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:41,649 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:41,649 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F1C5B015C6614631B4F092D734B8406D
2016-01-14 17:16:41,650 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:41,650 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 516C6AD6891D40139033401324E863D8
2016-01-14 17:16:41,650 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 516C6AD6891D40139033401324E863D8
2016-01-14 17:16:41,651 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:41,651 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: F1C5B015C6614631B4F092D734B8406D
2016-01-14 17:16:41,651 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F1C5B015C6614631B4F092D734B8406D
2016-01-14 17:16:41,652 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F7D408974AA14669B3E4B14DFE215782, ordinal: 0
2016-01-14 17:16:41,652 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F7D408974AA14669B3E4B14DFE215782, mem on start (mb), free: 299, total: 717, max: 717
2016-01-14 17:16:41,652 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:41,653 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:41,653 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:41,653 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:41,653 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:41,653 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:41,653 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:41,653 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:41,653 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:41,654 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 751828992, TotalAvailable: 503725424, TotalRequested/TotalJVMHeap:0.14
2016-01-14 17:16:41,654 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 516C6AD6891D40139033401324E863D8, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:41,654 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F1C5B015C6614631B4F092D734B8406D, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:41,654 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:41,654 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 516C6AD6891D40139033401324E863D8 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:41,654 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:41,655 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:41,655 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 516C6AD6891D40139033401324E863D8
2016-01-14 17:16:41,655 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:41,655 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 516C6AD6891D40139033401324E863D8
2016-01-14 17:16:41,655 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:41,655 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791801456_0001_1_00_000000_0
2016-01-14 17:16:41,655 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:41,656 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@53a9240
2016-01-14 17:16:41,657 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@53a9240
2016-01-14 17:16:41,657 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:41,657 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:41,658 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:41,659 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 516C6AD6891D40139033401324E863D8
2016-01-14 17:16:41,659 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: C6099D19F1A449EEA6D19C0C60E41431
2016-01-14 17:16:41,659 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] C6099D19F1A449EEA6D19C0C60E41431
2016-01-14 17:16:41,659 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@757b5022; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@72ca246f; comparator=cascading.tuple.hadoop.util.TupleComparator@351e2e49; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:41,659 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:41,660 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:41,660 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:41,672 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:41,673 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:41,673 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:41,673 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:41,673 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:41,674 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:41,674 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:41,674 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F7D408974AA14669B3E4B14DFE215782, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:41,676 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791801456_0001_1_00_000000_0
2016-01-14 17:16:41,676 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F7D408974AA14669B3E4B14DFE215782, mem on close (mb), free: 196, total: 717, max: 717
2016-01-14 17:16:41,676 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:41,676 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:41,677 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:41,677 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:41,679 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:41 Completed running task attempt: attempt_1452791801456_0001_1_00_000000_0
2016-01-14 17:16:41,682 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791801456_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:41,682 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751828992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791801674, Process_Duration=2, Process_End_Time=1452791801676, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:41,682 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791801456_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791801456_0001_1_00, eventCount=1
2016-01-14 17:16:41,682 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:41,683 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:41,683 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791801456_0001_00_000001
2016-01-14 17:16:41,684 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:41,686 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791801456_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F7D408974AA14669B3E4B14DFE215782, taskAttemptId=attempt_1452791801456_0001_1_00_000000_0, startTime=1452791801646, finishTime=1452791801684, timeTaken=38, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751828992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791801674, Process_Duration=2, Process_End_Time=1452791801676, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:41,686 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791801456_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:41,686 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:41,687 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791801456_0001_1_00_000000_0
2016-01-14 17:16:41,687 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:41,688 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791801456_0001_1][Event:TASK_FINISHED]: vertexName=F7D408974AA14669B3E4B14DFE215782, taskId=task_1452791801456_0001_1_00_000000, startTime=1452791801646, finishTime=1452791801687, timeTaken=41, status=SUCCEEDED, successfulAttemptID=attempt_1452791801456_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751828992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791801674, Process_Duration=2, Process_End_Time=1452791801676, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:41,688 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791801456_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:41,688 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791801456_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:41,688 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791801456_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791801456_0001_00_000001, stoppedTime=1452791801688, exitStatus=0
2016-01-14 17:16:41,688 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:41,689 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791801456_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:41,689 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791801456_0001_1_00 [F7D408974AA14669B3E4B14DFE215782] : 1
2016-01-14 17:16:41,689 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791801456_0001_1_00 [F7D408974AA14669B3E4B14DFE215782]
2016-01-14 17:16:41,690 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791801456_0001_1][Event:VERTEX_FINISHED]: vertexName=F7D408974AA14669B3E4B14DFE215782, vertexId=vertex_1452791801456_0001_1_00, initRequestedTime=1452791801619, initedTime=1452791801630, startRequestedTime=1452791801631, startedTime=1452791801631, finishTime=1452791801689, timeTaken=58, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751828992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791801674, Process_Duration=2, Process_End_Time=1452791801676, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791801646, firstTasksToStart=[ task_1452791801456_0001_1_00_000000 ], lastTaskFinishTime=1452791801684, lastTasksToFinish=[ task_1452791801456_0001_1_00_000000 ], minTaskDuration=38, maxTaskDuration=38, avgTaskDuration=38.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791801456_0001_1_00_000000 ], longestDurationTasks=[ task_1452791801456_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:41,690 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:41,690 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791801456_0001_1_00 [F7D408974AA14669B3E4B14DFE215782] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:41,691 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791801456_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:41,691 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791801456_0001_1_01 [F1C5B015C6614631B4F092D734B8406D] attempt: attempt_1452791801456_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:41,691 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: F1C5B015C6614631B4F092D734B8406D
2016-01-14 17:16:41,691 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791801456_0001_1_01 [F1C5B015C6614631B4F092D734B8406D]
2016-01-14 17:16:41,691 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791801456_0001_1_00 [F7D408974AA14669B3E4B14DFE215782] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:41,692 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791801456_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:41,692 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791801456_0001_1][Event:TASK_STARTED]: vertexName=F1C5B015C6614631B4F092D734B8406D, taskId=task_1452791801456_0001_1_01_000000, scheduledTime=1452791801692, launchTime=1452791801692
2016-01-14 17:16:41,692 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791801456_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:41,692 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:41,692 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791801456_0001_1_01 [F1C5B015C6614631B4F092D734B8406D]
2016-01-14 17:16:41,693 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [45B5FB3103C7450FB8A875B4FFE73693/C3EAE9C5835D4952BEBACEF814BE8695] every/(1/1) ...lyplatform/0_ARGS_RESULTS, VertexName: F1C5B015C6614631B4F092D734B8406D, VertexParallelism: 1, TaskAttemptID:attempt_1452791801456_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F7D408974AA14669B3E4B14DFE215782, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=03E55A9D901746C9B9225F912368B897, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:41,693 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791801456_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:41,693 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791801456_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:41,694 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791801456_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:41,694 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:41,694 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791801456_0001_00_000001, Exiting
2016-01-14 17:16:41,694 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791801456_0001_00_000001
2016-01-14 17:16:41,694 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791801456_0001_00_000001
2016-01-14 17:16:41,695 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791801456_0001_00_000002
2016-01-14 17:16:41,695 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791801456_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:41,696 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791801456_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:41,696 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791801456_0001_00_000001 completed successfully
2016-01-14 17:16:41,697 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791801456_0001_00_000002
2016-01-14 17:16:41,697 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791801456_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:41,697 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 17:16:41,699 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791801456_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:41,699 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:41,699 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791801456_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791801456_0001_00_000002, launchTime=1452791801699
2016-01-14 17:16:41,700 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791801456_0001_00_000002
2016-01-14 17:16:41,700 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791801456_0001_1_01_000000_0] to container: [container_1452791801456_0001_00_000002]
2016-01-14 17:16:41,700 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791801456_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:41,700 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791801456_0001_00_000002 given task: attempt_1452791801456_0001_1_01_000000_0
2016-01-14 17:16:41,700 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791801456_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791801456_0001_1_01_000000_0
2016-01-14 17:16:41,701 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791801456_0001_1_01_000000_0] started. Is using containerId: [container_1452791801456_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:41 Starting to run new task attempt: attempt_1452791801456_0001_1_01_000000_0
2016-01-14 17:16:41,701 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791801456_0001_1_01_000000_0
2016-01-14 17:16:41,701 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:41,701 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791801456_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F1C5B015C6614631B4F092D734B8406D, taskAttemptId=attempt_1452791801456_0001_1_01_000000_0, startTime=1452791801700, containerId=container_1452791801456_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791801456_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:41,701 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:41,702 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791801456_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:41,701 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:41,702 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:41,702 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791801456_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:41,702 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:41,703 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:41,703 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:41,703 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [45B5FB3103C7450FB8A875B4FFE73693/C3EAE9C5835D4952BEBACEF814BE8695] every/(1/1) ...lyplatform/0_ARGS_RESULTS, VertexName: F1C5B015C6614631B4F092D734B8406D, VertexParallelism: 1, TaskAttemptID:attempt_1452791801456_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F7D408974AA14669B3E4B14DFE215782, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=03E55A9D901746C9B9225F912368B897, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:41,704 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=751828992
2016-01-14 17:16:41,704 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791801456_0001_1_01_000000_0
2016-01-14 17:16:41,704 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:41,704 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:41,704 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791801456_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:41,705 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:41,705 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F7D408974AA14669B3E4B14DFE215782, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:41,705 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:41,705 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F7D408974AA14669B3E4B14DFE215782
2016-01-14 17:16:41,706 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 676646080, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:41,706 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:41,707 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 676646080, type: INPUT, componentVertexName: F7D408974AA14669B3E4B14DFE215782
2016-01-14 17:16:41,707 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F7D408974AA14669B3E4B14DFE215782
2016-01-14 17:16:41,707 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:41,708 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=03E55A9D901746C9B9225F912368B897, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:41,708 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:41,709 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 03E55A9D901746C9B9225F912368B897
2016-01-14 17:16:41,709 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:41,709 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 03E55A9D901746C9B9225F912368B897
2016-01-14 17:16:41,713 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F1C5B015C6614631B4F092D734B8406D, ordinal: 1
2016-01-14 17:16:41,713 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F1C5B015C6614631B4F092D734B8406D, mem on start (mb), free: 191, total: 717, max: 717
2016-01-14 17:16:41,713 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:41,713 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:41,713 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:41,716 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:41,716 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 03E55A9D901746C9B9225F912368B897
2016-01-14 17:16:41,717 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:41,717 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:41,717 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:41,717 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:41,717 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:41,717 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:41,717 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 676646080, TotalRequestedScaled: 6.245963815384616E8, TotalJVMHeap: 751828992, TotalAvailable: 503725424, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:41,718 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F7D408974AA14669B3E4B14DFE215782, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=676646080, allocated=503725424
2016-01-14 17:16:41,718 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 03E55A9D901746C9B9225F912368B897, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:41,718 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:41,718 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F7D408974AA14669B3E4B14DFE215782 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:41,718 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F7D408974AA14669B3E4B14DFE215782
2016-01-14 17:16:41,718 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:41,719 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:41,719 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:41,719 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:41,721 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: F7D408974AA14669B3E4B14DFE215782 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:41,721 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=676646080, postMergeMem=0, RuntimeTotalAvailable=503725424. Updated to: ShuffleMem=503725424, postMergeMem: 0
2016-01-14 17:16:41,722 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=503725424, maxSingleShuffleLimit=125931360, mergeThreshold=453352896, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:41,722 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:41,722 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F7D408974AA14669B3E4B14DFE215782
2016-01-14 17:16:41,724 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:41,725 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791801456_0001_1_01_000000_0
2016-01-14 17:16:41,725 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791801456_0001_1_00_000000_0_10213, runDuration: 31505, ]
2016-01-14 17:16:41,726 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791801456_0001_1_00_000000_0_10213] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791801726, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 17:16:41,726 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : F7D408974AA14669B3E4B14DFE215782
2016-01-14 17:16:41,727 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:41,727 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [F7D408974AA14669B3E4B14DFE215782] #36 in 2ms
2016-01-14 17:16:41,727 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [F7D408974AA14669B3E4B14DFE215782] #36, status:false, isInterrupted:false
2016-01-14 17:16:41,727 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:41,727 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:41,727 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:41,728 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:41,728 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:41,728 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:41,728 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : F7D408974AA14669B3E4B14DFE215782
2016-01-14 17:16:41,728 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:41,728 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: C6099D19F1A449EEA6D19C0C60E41431
2016-01-14 17:16:41,728 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_ARGS_RESULTS"], id: 03E55A9D901746C9B9225F912368B897
2016-01-14 17:16:41,729 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_ARGS_RESULTS"]
2016-01-14 17:16:41,729 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] C6099D19F1A449EEA6D19C0C60E41431, for 1 inputs
2016-01-14 17:16:41,729 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F1C5B015C6614631B4F092D734B8406D, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:41,730 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@3732eafc; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@6fe162b3
2016-01-14 17:16:41,734 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:41,734 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:41,734 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791801456_0001_1_01_000000_0
2016-01-14 17:16:41,734 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791801456_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:41,735 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791801456_0001_1_01_000000_0
2016-01-14 17:16:41,735 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F1C5B015C6614631B4F092D734B8406D, mem on close (mb), free: 187, total: 717, max: 717
2016-01-14 17:16:41,735 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: F7D408974AA14669B3E4B14DFE215782
2016-01-14 17:16:41,735 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:41,735 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:41,736 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:41,736 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:41,736 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:41 Completed running task attempt: attempt_1452791801456_0001_1_01_000000_0
2016-01-14 17:16:41,737 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791801456_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:41,737 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791801456_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F1C5B015C6614631B4F092D734B8406D, taskAttemptId=attempt_1452791801456_0001_1_01_000000_0, startTime=1452791801700, finishTime=1452791801737, timeTaken=37, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=652, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751828992, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791801729, Process_Duration=6, Process_End_Time=1452791801735, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:41,737 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791801456_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:41,738 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:41,738 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791801456_0001_1_01_000000_0
2016-01-14 17:16:41,738 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791801456_0001_1][Event:TASK_FINISHED]: vertexName=F1C5B015C6614631B4F092D734B8406D, taskId=task_1452791801456_0001_1_01_000000, startTime=1452791801700, finishTime=1452791801738, timeTaken=38, status=SUCCEEDED, successfulAttemptID=attempt_1452791801456_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=652, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751828992, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791801729, Process_Duration=6, Process_End_Time=1452791801735, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:41,738 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791801456_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:41,738 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791801456_0001_1_01 [F1C5B015C6614631B4F092D734B8406D] : 1
2016-01-14 17:16:41,738 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791801456_0001_1_01 [F1C5B015C6614631B4F092D734B8406D]
2016-01-14 17:16:41,739 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:41,739 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:41,739 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791801456_0001_1][Event:VERTEX_FINISHED]: vertexName=F1C5B015C6614631B4F092D734B8406D, vertexId=vertex_1452791801456_0001_1_01, initRequestedTime=1452791801620, initedTime=1452791801628, startRequestedTime=1452791801634, startedTime=1452791801634, finishTime=1452791801738, timeTaken=104, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=652, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751828992, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791801729, Process_Duration=6, Process_End_Time=1452791801735, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452791801700, firstTasksToStart=[ task_1452791801456_0001_1_01_000000 ], lastTaskFinishTime=1452791801737, lastTasksToFinish=[ task_1452791801456_0001_1_01_000000 ], minTaskDuration=37, maxTaskDuration=37, avgTaskDuration=37.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791801456_0001_1_01_000000 ], longestDurationTasks=[ task_1452791801456_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:41,739 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791801456_0001_1_01 [F1C5B015C6614631B4F092D734B8406D] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:41,740 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:41,740 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791801456_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:41,740 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791801456_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791801456_0001_00_000002, stoppedTime=1452791801740, exitStatus=0
2016-01-14 17:16:41,740 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791801456_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:41,740 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791801456_0001_1_01 [F1C5B015C6614631B4F092D734B8406D] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:41,740 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791801456_0001_1
2016-01-14 17:16:41,740 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791801456_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452791801456_0001_1
2016-01-14 17:16:41,740 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 03E55A9D901746C9B9225F912368B897 for vertex: vertex_1452791801456_0001_1_01 [F1C5B015C6614631B4F092D734B8406D]
2016-01-14 17:16:41,740 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:41,743 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=652, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751828992, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791801729, Process_Duration=6, Process_End_Time=1452791801735, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 17:16:41,744 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:41,744 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:41,745 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791801456_0001_00_000002
2016-01-14 17:16:41,745 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791801456_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 17:16:41,745 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791801456_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:41,745 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791801456_0001_00_000002, Exiting
2016-01-14 17:16:41,745 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791801456_0001_00_000002
2016-01-14 17:16:41,745 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791801456_0001_00_000002
2016-01-14 17:16:41,746 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791801456_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:41,746 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791801456_0001_00_000002 completed successfully
2016-01-14 17:16:41,749 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452791801456_0001_1_00 [F7D408974AA14669B3E4B14DFE215782]
2016-01-14 17:16:41,751 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791801456_0001_1, queueSize=0
2016-01-14 17:16:41,751 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:41,751 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791801456_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791801456_0001_1, startTime=1452791801619, finishTime=1452791801749, timeTaken=130, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=1505, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1503657984, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905583603403, Process_Duration=8, Process_End_Time=2905583603411, Read_Duration=2, Tuples_Read=40, Tuples_Written=40, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 17:16:41,751 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791801456_0001_1 finished with state: SUCCEEDED
2016-01-14 17:16:41,751 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791801456_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:41,751 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791801456_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:16:41,751 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452791801456_0001_00_000002
2016-01-14 17:16:41 Completed Dag: dag_1452791801456_0001_1
2016-01-14 17:16:41,752 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791801456_0001_1, dagState=SUCCEEDED
2016-01-14 17:16:41,752 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791801456_0001_1_post
2016-01-14 17:16:41,752 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:41,752 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:41,759 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...lyplatform/0_ARGS_RESULTS, applicationId=application_1452791801456_0001
2016-01-14 17:16:41,761 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:41,761 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:41,761 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:41,761 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:41,761 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:41,762 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:41,762 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:41,762 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:41,762 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:41,762 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:41,762 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:41,762 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:41,763 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:41,767 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/45B5FB3103C7450FB8A875B4FFE73693/C3EAE9C5835D4952BEBACEF814BE8695/.tez/application_1452791801456_0001
2016-01-14 17:16:41,768 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.330
2016-01-14 17:16:41,846 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_ARGS_RESULTS
2016-01-14 17:16:41,868 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:41,868 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:41,892 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.002
2016-01-14 17:16:41,894 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.004
2016-01-14 17:16:41,894 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 17:16:41,895 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 17:16:41,895 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_ARGS_ALL
2016-01-14 17:16:41,913 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:41,913 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:41,940 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.006
2016-01-14 17:16:41,950 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.016
2016-01-14 17:16:41,951 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:41,951 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:41,952 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:41,965 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:41,970 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_ARGS_REPLACE
2016-01-14 17:16:42,014 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:42,014 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:42,014 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_ARGS_REPLACE"]
2016-01-14 17:16:42,014 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:42,014 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:42,014 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:42,015 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...lyplatform/0_ARGS_REPLACE
2016-01-14 17:16:42,015 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:42,025 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:42,029 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:42,029 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:42,029 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:42,039 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/2D79D35C1AFE41518849785F16434191/D366EF6FFCC4499AB05FFD3467926F08/.tez/application_1452791802029_0001 doesn't exist and is created
2016-01-14 17:16:42,054 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:42,055 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:42,055 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/2D79D35C1AFE41518849785F16434191/D366EF6FFCC4499AB05FFD3467926F08/.tez/application_1452791802029_0001
2016-01-14 17:16:42,058 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791802029_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:42,069 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:42,070 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:42,071 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:42,071 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:42,071 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/2D79D35C1AFE41518849785F16434191/D366EF6FFCC4499AB05FFD3467926F08/.tez/application_1452791802029_0001/localmode-log-dir
2016-01-14 17:16:42,072 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/2D79D35C1AFE41518849785F16434191/D366EF6FFCC4499AB05FFD3467926F08/.tez/application_1452791802029_0001/localmode-log-dir/history.txt.appattempt_1452791802029_0001_000000, maxErrors=10
2016-01-14 17:16:42,072 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:42,072 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791802029_0001_000000, appSubmitTime=1452791802058, launchTime=1452791802058
2016-01-14 17:16:42,074 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:42,075 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:42,077 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:42,078 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:42,079 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:48829
2016-01-14 17:16:42,082 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:42,083 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791802029_0001_000000, startTime=1452791802083
2016-01-14 17:16:42,083 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:42,083 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:42,155 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:42,155 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:42,155 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...lyplatform/0_ARGS_REPLACE, applicationId=application_1452791802029_0001, dagName=[2D79D35C1AFE41518849785F16434191/D366EF6FFCC4499AB05FFD3467926F08] every/(1/1) ...lyplatform/0_ARGS_REPLACE
2016-01-14 17:16:42,170 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [2D79D35C1AFE41518849785F16434191/D366EF6FFCC4499AB05FFD3467926F08] every/(1/1) ...lyplatform/0_ARGS_REPLACE
2016-01-14 17:16:42,171 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791802029_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/2D79D35C1AFE41518849785F16434191/D366EF6FFCC4499AB05FFD3467926F08/.tez/application_1452791802029_0001/localmode-log-dir/dag_1452791802029_0001_1.dot
2016-01-14 17:16:42,171 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/2D79D35C1AFE41518849785F16434191/D366EF6FFCC4499AB05FFD3467926F08/.tez/application_1452791802029_0001/localmode-log-dir/dag_1452791802029_0001_1-tez-dag.pb.txt
2016-01-14 17:16:42,178 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791802029_0001_1
2016-01-14 17:16:42,178 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:42,179 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [2D79D35C1AFE41518849785F16434191/D366EF6FFCC4499AB05FFD3467926F08] every/(1/1) ...lyplatform/0_ARGS_REPLACE
2016-01-14 17:16:42 Running Dag: dag_1452791802029_0001_1
2016-01-14 17:16:42,190 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791802029_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791802029_0001_1, submitTime=1452791802170
2016-01-14 17:16:42,190 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:42,191 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex C720FCE13ED44183BCBD34885AA35A16
2016-01-14 17:16:42,192 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:42,192 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791802029_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791802029_0001_1, initTime=1452791802191
2016-01-14 17:16:42,192 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:42,192 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791802029_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:42,193 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791802029_0001_1][Event:DAG_STARTED]: dagID=dag_1452791802029_0001_1, startTime=1452791802193
2016-01-14 17:16:42,193 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:42,193 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:42,193 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791802029_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:42,193 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: CA77778C247C426A83165A1E84FB0339 : {F8C922EA4A5D4E29AA66D51E3F200B01={InputName=F8C922EA4A5D4E29AA66D51E3F200B01}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:42,193 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: F8C922EA4A5D4E29AA66D51E3F200B01, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:42,193 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791802029_0001_1_00 [CA77778C247C426A83165A1E84FB0339]
2016-01-14 17:16:42,193 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791802029_0001_1_00 [CA77778C247C426A83165A1E84FB0339]
2016-01-14 17:16:42,193 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791802029_0001_1_00 [CA77778C247C426A83165A1E84FB0339]
2016-01-14 17:16:42,193 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...lyplatform/0_ARGS_REPLACE, applicationId=application_1452791802029_0001, dagName=[2D79D35C1AFE41518849785F16434191/D366EF6FFCC4499AB05FFD3467926F08] every/(1/1) ...lyplatform/0_ARGS_REPLACE
2016-01-14 17:16:42,194 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791802029_0001_1_00 [CA77778C247C426A83165A1E84FB0339]. Starting root input initializers: 1
2016-01-14 17:16:42,194 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791802029_0001_1_00 [CA77778C247C426A83165A1E84FB0339] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:42,194 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791802029_0001_1_01 [C720FCE13ED44183BCBD34885AA35A16]
2016-01-14 17:16:42,197 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: F8C922EA4A5D4E29AA66D51E3F200B01 on vertex vertex_1452791802029_0001_1_00 [CA77778C247C426A83165A1E84FB0339]
2016-01-14 17:16:42,197 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:42,197 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791802029_0001_1_01 [C720FCE13ED44183BCBD34885AA35A16]
2016-01-14 17:16:42,197 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791802029_0001_1_01 [C720FCE13ED44183BCBD34885AA35A16]
2016-01-14 17:16:42,198 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791802029_0001_1_01 [C720FCE13ED44183BCBD34885AA35A16]
2016-01-14 17:16:42,198 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=3E8D3D6B2FF4472DB2C752618C7DF11B, vertexId=vertex_1452791802029_0001_1_01 [C720FCE13ED44183BCBD34885AA35A16], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:42,198 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=3E8D3D6B2FF4472DB2C752618C7DF11B, vertexId=vertex_1452791802029_0001_1_01 [C720FCE13ED44183BCBD34885AA35A16]
2016-01-14 17:16:42,199 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:42,199 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input F8C922EA4A5D4E29AA66D51E3F200B01 asking for 1 tasks. Headroom: 716 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:42,199 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 17:16:42,200 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 17:16:42,200 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:42,200 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=3E8D3D6B2FF4472DB2C752618C7DF11B, vertexName=C720FCE13ED44183BCBD34885AA35A16, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:42,200 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:42,200 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=3E8D3D6B2FF4472DB2C752618C7DF11B, vertexId=vertex_1452791802029_0001_1_01 [C720FCE13ED44183BCBD34885AA35A16]
2016-01-14 17:16:42,202 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:42,202 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: F8C922EA4A5D4E29AA66D51E3F200B01 on vertex vertex_1452791802029_0001_1_00 [CA77778C247C426A83165A1E84FB0339]
2016-01-14 17:16:42,207 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:42,207 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791802029_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C720FCE13ED44183BCBD34885AA35A16, vertexId=vertex_1452791802029_0001_1_01, initRequestedTime=1452791802194, initedTime=1452791802206, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:42,207 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791802029_0001_1_01 [C720FCE13ED44183BCBD34885AA35A16] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:42,207 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791802029_0001_1_00 [CA77778C247C426A83165A1E84FB0339] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:42,207 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {F8C922EA4A5D4E29AA66D51E3F200B01=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:42,208 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791802029_0001_1_00 [CA77778C247C426A83165A1E84FB0339] parallelism set to 1
2016-01-14 17:16:42,208 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791802029_0001_1_00 [CA77778C247C426A83165A1E84FB0339] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:42,208 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791802029_0001, with dag id: dag_1452791802029_0001_1
2016-01-14 17:16:42,208 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791802029_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791802029_0001_1_00, eventCount=1
2016-01-14 17:16:42,209 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:42,209 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791802029_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CA77778C247C426A83165A1E84FB0339, vertexId=vertex_1452791802029_0001_1_00, initRequestedTime=1452791802193, initedTime=1452791802208, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:42,209 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791802029_0001_1_00 [CA77778C247C426A83165A1E84FB0339] with distanceFromRoot: 0
2016-01-14 17:16:42,209 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791802029_0001_1_00 [CA77778C247C426A83165A1E84FB0339] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:42,209 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in CA77778C247C426A83165A1E84FB0339
2016-01-14 17:16:42,209 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791802029_0001_1_00 [CA77778C247C426A83165A1E84FB0339]
2016-01-14 17:16:42,209 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:42,210 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791802029_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791802029_0001_1_00, startRequestedTime=1452791802209, startedTime=1452791802209
2016-01-14 17:16:42,210 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791802029_0001_1_00 [CA77778C247C426A83165A1E84FB0339] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:42,210 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:42,210 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791802029_0001_1][Event:TASK_STARTED]: vertexName=CA77778C247C426A83165A1E84FB0339, taskId=task_1452791802029_0001_1_00_000000, scheduledTime=1452791802210, launchTime=1452791802210
2016-01-14 17:16:42,210 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791802029_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:42,210 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791802029_0001_1_00 for vertex: vertex_1452791802029_0001_1_01 [C720FCE13ED44183BCBD34885AA35A16] numStartedSources: 1 numSources: 1
2016-01-14 17:16:42,210 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791802029_0001_1_01 [C720FCE13ED44183BCBD34885AA35A16] with distanceFromRoot: 1
2016-01-14 17:16:42,211 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791802029_0001_1_00 [CA77778C247C426A83165A1E84FB0339]
2016-01-14 17:16:42,211 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: CA77778C247C426A83165A1E84FB0339 in vertex: C720FCE13ED44183BCBD34885AA35A16
2016-01-14 17:16:42,211 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: C720FCE13ED44183BCBD34885AA35A16 with 1 source tasks and 1 pending tasks
2016-01-14 17:16:42,212 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:42,212 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791802029_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791802029_0001_1_01, startRequestedTime=1452791802211, startedTime=1452791802211
2016-01-14 17:16:42,212 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791802029_0001_1_01 [C720FCE13ED44183BCBD34885AA35A16] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:42,213 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [2D79D35C1AFE41518849785F16434191/D366EF6FFCC4499AB05FFD3467926F08] every/(1/1) ...lyplatform/0_ARGS_REPLACE, VertexName: CA77778C247C426A83165A1E84FB0339, VertexParallelism: 1, TaskAttemptID:attempt_1452791802029_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F8C922EA4A5D4E29AA66D51E3F200B01, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C720FCE13ED44183BCBD34885AA35A16, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:42,213 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791802029_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:42,213 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:42,214 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:42,214 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:42,214 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:42,215 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791802029_0001_00_000001
2016-01-14 17:16:42,215 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791802029_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:42,217 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791802029_0001_00_000001
2016-01-14 17:16:42,217 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:42,217 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791802029_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:42,217 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:42,217 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791802029_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791802029_0001_00_000001, launchTime=1452791802217
2016-01-14 17:16:42,218 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791802029_0001_00_000001
2016-01-14 17:16:42,218 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791802029_0001_1_00_000000_0] to container: [container_1452791802029_0001_00_000001]
2016-01-14 17:16:42,218 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791802029_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:42,219 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791802029_0001_00_000001 given task: attempt_1452791802029_0001_1_00_000000_0
2016-01-14 17:16:42,219 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791802029_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791802029_0001_1_00_000000_0
2016-01-14 17:16:42,219 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791802029_0001_1_00_000000_0] started. Is using containerId: [container_1452791802029_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:42,219 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791802029_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CA77778C247C426A83165A1E84FB0339, taskAttemptId=attempt_1452791802029_0001_1_00_000000_0, startTime=1452791802219, containerId=container_1452791802029_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791802029_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:42,219 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:42,219 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791802029_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:42,219 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791802029_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:42 Starting to run new task attempt: attempt_1452791802029_0001_1_00_000000_0
2016-01-14 17:16:42,220 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791802029_0001_1_00_000000_0
2016-01-14 17:16:42,220 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:42,220 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:42,220 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:42,220 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:42,220 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:42,220 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:42,220 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [2D79D35C1AFE41518849785F16434191/D366EF6FFCC4499AB05FFD3467926F08] every/(1/1) ...lyplatform/0_ARGS_REPLACE, VertexName: CA77778C247C426A83165A1E84FB0339, VertexParallelism: 1, TaskAttemptID:attempt_1452791802029_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F8C922EA4A5D4E29AA66D51E3F200B01, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C720FCE13ED44183BCBD34885AA35A16, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:42,221 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=751304704
2016-01-14 17:16:42,221 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791802029_0001_1_00_000000_0
2016-01-14 17:16:42,221 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:42,221 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:42,221 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F8C922EA4A5D4E29AA66D51E3F200B01, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:42,222 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:42,222 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F8C922EA4A5D4E29AA66D51E3F200B01
2016-01-14 17:16:42,222 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791802029_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:42,222 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=C720FCE13ED44183BCBD34885AA35A16, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:42,222 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:42,223 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:42,223 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: C720FCE13ED44183BCBD34885AA35A16
2016-01-14 17:16:42,222 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: F8C922EA4A5D4E29AA66D51E3F200B01
2016-01-14 17:16:42,224 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:42,224 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: C720FCE13ED44183BCBD34885AA35A16
2016-01-14 17:16:42,225 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: C720FCE13ED44183BCBD34885AA35A16
2016-01-14 17:16:42,225 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:42,225 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:42,225 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: F8C922EA4A5D4E29AA66D51E3F200B01
2016-01-14 17:16:42,226 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F8C922EA4A5D4E29AA66D51E3F200B01
2016-01-14 17:16:42,229 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CA77778C247C426A83165A1E84FB0339, ordinal: 0
2016-01-14 17:16:42,229 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CA77778C247C426A83165A1E84FB0339, mem on start (mb), free: 271, total: 716, max: 716
2016-01-14 17:16:42,229 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:42,229 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:42,230 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:42,230 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:42,230 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:42,230 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:42,230 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:42,230 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:42,231 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:42,231 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 751304704, TotalAvailable: 503374151, TotalRequested/TotalJVMHeap:0.14
2016-01-14 17:16:42,231 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F8C922EA4A5D4E29AA66D51E3F200B01, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:42,231 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, C720FCE13ED44183BCBD34885AA35A16, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:42,231 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:42,231 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F8C922EA4A5D4E29AA66D51E3F200B01 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:42,231 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F8C922EA4A5D4E29AA66D51E3F200B01
2016-01-14 17:16:42,232 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:42,232 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F8C922EA4A5D4E29AA66D51E3F200B01
2016-01-14 17:16:42,232 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:42,232 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:42,232 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:42,233 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791802029_0001_1_00_000000_0
2016-01-14 17:16:42,235 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:42,235 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:42,235 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: F8C922EA4A5D4E29AA66D51E3F200B01
2016-01-14 17:16:42,236 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 6876A87EBA0B4BCA9BF0B8CC0F828680
2016-01-14 17:16:42,236 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 6876A87EBA0B4BCA9BF0B8CC0F828680
2016-01-14 17:16:42,236 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@1dcd082c
2016-01-14 17:16:42,236 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@1a922af4; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@724206cc; comparator=cascading.tuple.hadoop.util.TupleComparator@4ccc75ae; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:42,236 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:42,236 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:42,237 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:42,237 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@1dcd082c
2016-01-14 17:16:42,239 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:42,239 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:42,344 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:42,345 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:42,345 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:42,345 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:42,345 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:42,346 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:42,346 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:42,346 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CA77778C247C426A83165A1E84FB0339, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:42,349 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791802029_0001_1_00_000000_0
2016-01-14 17:16:42,349 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CA77778C247C426A83165A1E84FB0339, mem on close (mb), free: 168, total: 716, max: 716
2016-01-14 17:16:42,350 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:42,350 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:42,350 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:42,350 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:42,352 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:42 Completed running task attempt: attempt_1452791802029_0001_1_00_000000_0
2016-01-14 17:16:42,355 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791802029_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:42,355 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751304704, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791802346, Process_Duration=3, Process_End_Time=1452791802349, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:42,355 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791802029_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791802029_0001_1_00, eventCount=1
2016-01-14 17:16:42,356 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:42,356 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:42,356 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791802029_0001_00_000001
2016-01-14 17:16:42,356 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:42,356 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791802029_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CA77778C247C426A83165A1E84FB0339, taskAttemptId=attempt_1452791802029_0001_1_00_000000_0, startTime=1452791802219, finishTime=1452791802356, timeTaken=137, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751304704, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791802346, Process_Duration=3, Process_End_Time=1452791802349, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:42,356 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791802029_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:42,356 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:42,356 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791802029_0001_1_00_000000_0
2016-01-14 17:16:42,357 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:42,357 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791802029_0001_1][Event:TASK_FINISHED]: vertexName=CA77778C247C426A83165A1E84FB0339, taskId=task_1452791802029_0001_1_00_000000, startTime=1452791802219, finishTime=1452791802357, timeTaken=138, status=SUCCEEDED, successfulAttemptID=attempt_1452791802029_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751304704, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791802346, Process_Duration=3, Process_End_Time=1452791802349, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:42,357 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791802029_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:42,357 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791802029_0001_1_00 [CA77778C247C426A83165A1E84FB0339] : 1
2016-01-14 17:16:42,358 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791802029_0001_1_00 [CA77778C247C426A83165A1E84FB0339]
2016-01-14 17:16:42,358 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:42,358 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791802029_0001_1][Event:VERTEX_FINISHED]: vertexName=CA77778C247C426A83165A1E84FB0339, vertexId=vertex_1452791802029_0001_1_00, initRequestedTime=1452791802193, initedTime=1452791802208, startRequestedTime=1452791802209, startedTime=1452791802209, finishTime=1452791802358, timeTaken=149, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751304704, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791802346, Process_Duration=3, Process_End_Time=1452791802349, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791802219, firstTasksToStart=[ task_1452791802029_0001_1_00_000000 ], lastTaskFinishTime=1452791802356, lastTasksToFinish=[ task_1452791802029_0001_1_00_000000 ], minTaskDuration=137, maxTaskDuration=137, avgTaskDuration=137.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791802029_0001_1_00_000000 ], longestDurationTasks=[ task_1452791802029_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:42,358 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791802029_0001_1_00 [CA77778C247C426A83165A1E84FB0339] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:42,359 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791802029_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:42,359 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791802029_0001_1_01 [C720FCE13ED44183BCBD34885AA35A16] attempt: attempt_1452791802029_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:42,359 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: C720FCE13ED44183BCBD34885AA35A16
2016-01-14 17:16:42,359 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791802029_0001_1_01 [C720FCE13ED44183BCBD34885AA35A16]
2016-01-14 17:16:42,359 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791802029_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791802029_0001_00_000001, stoppedTime=1452791802359, exitStatus=0
2016-01-14 17:16:42,359 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791802029_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:42,359 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:42,359 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791802029_0001_1_00 [CA77778C247C426A83165A1E84FB0339] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:42,360 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791802029_0001_1][Event:TASK_STARTED]: vertexName=C720FCE13ED44183BCBD34885AA35A16, taskId=task_1452791802029_0001_1_01_000000, scheduledTime=1452791802360, launchTime=1452791802360
2016-01-14 17:16:42,361 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791802029_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:42,361 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791802029_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:42,361 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791802029_0001_1_01 [C720FCE13ED44183BCBD34885AA35A16]
2016-01-14 17:16:42,362 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791802029_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:42,362 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [2D79D35C1AFE41518849785F16434191/D366EF6FFCC4499AB05FFD3467926F08] every/(1/1) ...lyplatform/0_ARGS_REPLACE, VertexName: C720FCE13ED44183BCBD34885AA35A16, VertexParallelism: 1, TaskAttemptID:attempt_1452791802029_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=CA77778C247C426A83165A1E84FB0339, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=3E8D3D6B2FF4472DB2C752618C7DF11B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:42,362 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791802029_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:42,360 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791802029_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:42,360 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:42,362 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:42,363 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791802029_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:42,363 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791802029_0001_00_000002
2016-01-14 17:16:42,363 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791802029_0001_00_000001, Exiting
2016-01-14 17:16:42,364 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791802029_0001_00_000001
2016-01-14 17:16:42,364 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791802029_0001_00_000001
2016-01-14 17:16:42,364 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791802029_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:42,365 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791802029_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:42,365 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791802029_0001_00_000001 completed successfully
2016-01-14 17:16:42,366 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791802029_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:42,366 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791802029_0001_00_000002
2016-01-14 17:16:42,367 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 17:16:42,369 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791802029_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:42,369 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:42,369 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791802029_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791802029_0001_00_000002, launchTime=1452791802369
2016-01-14 17:16:42,370 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791802029_0001_00_000002
2016-01-14 17:16:42,370 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791802029_0001_1_01_000000_0] to container: [container_1452791802029_0001_00_000002]
2016-01-14 17:16:42,370 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791802029_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:42,371 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791802029_0001_1_01_000000_0] started. Is using containerId: [container_1452791802029_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:42,371 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791802029_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C720FCE13ED44183BCBD34885AA35A16, taskAttemptId=attempt_1452791802029_0001_1_01_000000_0, startTime=1452791802371, containerId=container_1452791802029_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791802029_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:42,371 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:42,371 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791802029_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:42,372 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791802029_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:42,371 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791802029_0001_00_000002 given task: attempt_1452791802029_0001_1_01_000000_0
2016-01-14 17:16:42,372 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791802029_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791802029_0001_1_01_000000_0
2016-01-14 17:16:42 Starting to run new task attempt: attempt_1452791802029_0001_1_01_000000_0
2016-01-14 17:16:42,372 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791802029_0001_1_01_000000_0
2016-01-14 17:16:42,372 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:42,372 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:42,372 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:42,373 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:42,373 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:42,373 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:42,373 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [2D79D35C1AFE41518849785F16434191/D366EF6FFCC4499AB05FFD3467926F08] every/(1/1) ...lyplatform/0_ARGS_REPLACE, VertexName: C720FCE13ED44183BCBD34885AA35A16, VertexParallelism: 1, TaskAttemptID:attempt_1452791802029_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=CA77778C247C426A83165A1E84FB0339, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=3E8D3D6B2FF4472DB2C752618C7DF11B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:42,373 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=751304704
2016-01-14 17:16:42,373 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791802029_0001_1_01_000000_0
2016-01-14 17:16:42,373 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:42,373 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:42,374 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:42,374 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=CA77778C247C426A83165A1E84FB0339, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:42,374 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:42,374 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: CA77778C247C426A83165A1E84FB0339
2016-01-14 17:16:42,376 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 676174208, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:42,376 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:42,376 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 676174208, type: INPUT, componentVertexName: CA77778C247C426A83165A1E84FB0339
2016-01-14 17:16:42,376 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: CA77778C247C426A83165A1E84FB0339
2016-01-14 17:16:42,377 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:42,377 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791802029_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:42,378 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=3E8D3D6B2FF4472DB2C752618C7DF11B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:42,379 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:42,379 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 3E8D3D6B2FF4472DB2C752618C7DF11B
2016-01-14 17:16:42,379 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:42,379 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 3E8D3D6B2FF4472DB2C752618C7DF11B
2016-01-14 17:16:42,391 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C720FCE13ED44183BCBD34885AA35A16, ordinal: 1
2016-01-14 17:16:42,391 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C720FCE13ED44183BCBD34885AA35A16, mem on start (mb), free: 162, total: 716, max: 716
2016-01-14 17:16:42,391 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:42,392 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:42,392 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:42,394 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:42,394 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 3E8D3D6B2FF4472DB2C752618C7DF11B
2016-01-14 17:16:42,394 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:42,395 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:42,395 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:42,395 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:42,396 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:42,396 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:42,397 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 676174208, TotalRequestedScaled: 6.241608073846154E8, TotalJVMHeap: 751304704, TotalAvailable: 503374151, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:42,397 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, CA77778C247C426A83165A1E84FB0339, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=676174208, allocated=503374151
2016-01-14 17:16:42,397 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 3E8D3D6B2FF4472DB2C752618C7DF11B, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:42,398 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:42,398 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: CA77778C247C426A83165A1E84FB0339 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:42,398 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: CA77778C247C426A83165A1E84FB0339
2016-01-14 17:16:42,399 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:42,399 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:42,399 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:42,400 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:42,400 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: CA77778C247C426A83165A1E84FB0339 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:42,400 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=676174208, postMergeMem=0, RuntimeTotalAvailable=503374151. Updated to: ShuffleMem=503374151, postMergeMem: 0
2016-01-14 17:16:42,400 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=503374151, maxSingleShuffleLimit=125843536, mergeThreshold=453036704, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:42,400 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:42,402 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: CA77778C247C426A83165A1E84FB0339
2016-01-14 17:16:42,403 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:42,404 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791802029_0001_1_01_000000_0
2016-01-14 17:16:42,405 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791802029_0001_1_00_000000_0_10219, runDuration: 131189, ]
2016-01-14 17:16:42,406 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791802029_0001_1_00_000000_0_10219] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791802406, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 17:16:42,406 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : CA77778C247C426A83165A1E84FB0339
2016-01-14 17:16:42,406 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:42,406 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [CA77778C247C426A83165A1E84FB0339] #37 in 1ms
2016-01-14 17:16:42,406 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [CA77778C247C426A83165A1E84FB0339] #37, status:false, isInterrupted:false
2016-01-14 17:16:42,407 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:42,407 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:42,407 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:42,407 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:42,407 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:42,407 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : CA77778C247C426A83165A1E84FB0339
2016-01-14 17:16:42,408 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:42,409 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:42,409 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 6876A87EBA0B4BCA9BF0B8CC0F828680
2016-01-14 17:16:42,410 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_ARGS_REPLACE"], id: 3E8D3D6B2FF4472DB2C752618C7DF11B
2016-01-14 17:16:42,410 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_ARGS_REPLACE"]
2016-01-14 17:16:42,410 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 6876A87EBA0B4BCA9BF0B8CC0F828680, for 1 inputs
2016-01-14 17:16:42,411 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C720FCE13ED44183BCBD34885AA35A16, all 1 inputs ready in: 00:00:00.001
2016-01-14 17:16:42,411 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@57a6c2e5; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@5457558
2016-01-14 17:16:42,415 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:42,415 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:42,416 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791802029_0001_1_01_000000_0
2016-01-14 17:16:42,416 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791802029_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:42,416 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791802029_0001_1_01_000000_0
2016-01-14 17:16:42,417 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C720FCE13ED44183BCBD34885AA35A16, mem on close (mb), free: 158, total: 716, max: 716
2016-01-14 17:16:42,417 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: CA77778C247C426A83165A1E84FB0339
2016-01-14 17:16:42,417 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:42,417 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:42,417 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:42,418 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:42,418 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:42 Completed running task attempt: attempt_1452791802029_0001_1_01_000000_0
2016-01-14 17:16:42,419 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791802029_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:42,419 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791802029_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C720FCE13ED44183BCBD34885AA35A16, taskAttemptId=attempt_1452791802029_0001_1_01_000000_0, startTime=1452791802371, finishTime=1452791802419, timeTaken=48, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=183, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751304704, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791802411, Process_Duration=5, Process_End_Time=1452791802416, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:42,419 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791802029_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:42,419 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791802029_0001_1_01_000000_0
2016-01-14 17:16:42,420 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:42,420 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791802029_0001_1][Event:TASK_FINISHED]: vertexName=C720FCE13ED44183BCBD34885AA35A16, taskId=task_1452791802029_0001_1_01_000000, startTime=1452791802371, finishTime=1452791802419, timeTaken=48, status=SUCCEEDED, successfulAttemptID=attempt_1452791802029_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=183, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751304704, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791802411, Process_Duration=5, Process_End_Time=1452791802416, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:42,420 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791802029_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:42,420 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791802029_0001_1_01 [C720FCE13ED44183BCBD34885AA35A16] : 1
2016-01-14 17:16:42,420 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791802029_0001_1_01 [C720FCE13ED44183BCBD34885AA35A16]
2016-01-14 17:16:42,420 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:42,421 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791802029_0001_1][Event:VERTEX_FINISHED]: vertexName=C720FCE13ED44183BCBD34885AA35A16, vertexId=vertex_1452791802029_0001_1_01, initRequestedTime=1452791802194, initedTime=1452791802206, startRequestedTime=1452791802211, startedTime=1452791802211, finishTime=1452791802420, timeTaken=209, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=183, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751304704, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791802411, Process_Duration=5, Process_End_Time=1452791802416, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452791802371, firstTasksToStart=[ task_1452791802029_0001_1_01_000000 ], lastTaskFinishTime=1452791802419, lastTasksToFinish=[ task_1452791802029_0001_1_01_000000 ], minTaskDuration=48, maxTaskDuration=48, avgTaskDuration=48.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791802029_0001_1_01_000000 ], longestDurationTasks=[ task_1452791802029_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:42,421 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:42,421 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791802029_0001_1_01 [C720FCE13ED44183BCBD34885AA35A16] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:42,421 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791802029_0001_1_01 [C720FCE13ED44183BCBD34885AA35A16] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:42,421 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791802029_0001_1
2016-01-14 17:16:42,421 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:42,421 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791802029_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452791802029_0001_1
2016-01-14 17:16:42,421 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452791802029_0001_1_00 [CA77778C247C426A83165A1E84FB0339]
2016-01-14 17:16:42,421 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 3E8D3D6B2FF4472DB2C752618C7DF11B for vertex: vertex_1452791802029_0001_1_01 [C720FCE13ED44183BCBD34885AA35A16]
2016-01-14 17:16:42,431 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791802029_0001_1, queueSize=0
2016-01-14 17:16:42,431 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:42,432 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791802029_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791802029_0001_1, startTime=1452791802193, finishTime=1452791802430, timeTaken=237, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=516, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1502609408, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905583604757, Process_Duration=8, Process_End_Time=2905583604765, Read_Duration=3, Tuples_Read=40, Tuples_Written=40, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 17:16:42,432 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791802029_0001_1 finished with state: SUCCEEDED
2016-01-14 17:16:42,432 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791802029_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:42,433 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=183, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751304704, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791802411, Process_Duration=5, Process_End_Time=1452791802416, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 17:16:42,433 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791802029_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:42,433 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:42,433 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791802029_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791802029_0001_00_000002, stoppedTime=1452791802433, exitStatus=0
2016-01-14 17:16:42,433 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:42,433 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:42,433 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791802029_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:42,434 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791802029_0001_00_000002
2016-01-14 17:16:42 Completed Dag: dag_1452791802029_0001_1
2016-01-14 17:16:42,434 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791802029_0001_1, dagState=SUCCEEDED
2016-01-14 17:16:42,434 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791802029_0001_1_post
2016-01-14 17:16:42,434 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:42,434 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:42,434 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791802029_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 17:16:42,434 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791802029_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:42,434 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791802029_0001_00_000002. Relying on regular task shutdown for it to end
2016-01-14 17:16:42,434 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791802029_0001_00_000002, Exiting
2016-01-14 17:16:42,434 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791802029_0001_00_000002
2016-01-14 17:16:42,434 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791802029_0001_00_000002
2016-01-14 17:16:42,435 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791802029_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:42,435 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791802029_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:42,435 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791802029_0001_00_000002 completed successfully
2016-01-14 17:16:42,435 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791802029_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:42,439 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...lyplatform/0_ARGS_REPLACE, applicationId=application_1452791802029_0001
2016-01-14 17:16:42,440 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:42,441 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:42,441 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:42,441 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:42,441 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:42,441 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:42,442 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:42,442 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:42,442 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:42,442 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:42,442 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:42,442 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:42,443 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:42,447 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/2D79D35C1AFE41518849785F16434191/D366EF6FFCC4499AB05FFD3467926F08/.tez/application_1452791802029_0001
2016-01-14 17:16:42,448 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.434
2016-01-14 17:16:42,519 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_ARGS_REPLACE
2016-01-14 17:16:42,539 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:42,539 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:42,577 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-01-14 17:16:42,579 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.016
2016-01-14 17:16:42,579 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:42,579 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:42,579 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:42,592 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:42,597 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_ARGS_SWAP
2016-01-14 17:16:42,638 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:42,638 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:42,638 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_ARGS_SWAP"]
2016-01-14 17:16:42,638 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:42,638 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:42,638 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:42,639 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...emblyplatform/0_ARGS_SWAP
2016-01-14 17:16:42,639 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:42,650 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:42,654 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:42,654 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:42,655 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:42,664 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/50A3138F1DCE43568600C939AA509404/D2F91A1AF5204329886E5EFA223DEC90/.tez/application_1452791802654_0001 doesn't exist and is created
2016-01-14 17:16:42,679 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:42,680 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:42,680 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/50A3138F1DCE43568600C939AA509404/D2F91A1AF5204329886E5EFA223DEC90/.tez/application_1452791802654_0001
2016-01-14 17:16:42,682 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791802654_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:42,691 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:42,692 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:42,693 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:42,693 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:42,693 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/50A3138F1DCE43568600C939AA509404/D2F91A1AF5204329886E5EFA223DEC90/.tez/application_1452791802654_0001/localmode-log-dir
2016-01-14 17:16:42,693 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/50A3138F1DCE43568600C939AA509404/D2F91A1AF5204329886E5EFA223DEC90/.tez/application_1452791802654_0001/localmode-log-dir/history.txt.appattempt_1452791802654_0001_000000, maxErrors=10
2016-01-14 17:16:42,693 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:42,693 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791802654_0001_000000, appSubmitTime=1452791802682, launchTime=1452791802682
2016-01-14 17:16:42,697 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:42,698 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:42,698 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:42,701 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:42,701 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:34225
2016-01-14 17:16:42,704 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:42,705 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791802654_0001_000000, startTime=1452791802705
2016-01-14 17:16:42,705 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:42,705 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:42,739 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 17:16:42,739 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 17:16:42,780 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:42,780 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:42,780 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...emblyplatform/0_ARGS_SWAP, applicationId=application_1452791802654_0001, dagName=[50A3138F1DCE43568600C939AA509404/D2F91A1AF5204329886E5EFA223DEC90] every/(1/1) ...emblyplatform/0_ARGS_SWAP
2016-01-14 17:16:42,794 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [50A3138F1DCE43568600C939AA509404/D2F91A1AF5204329886E5EFA223DEC90] every/(1/1) ...emblyplatform/0_ARGS_SWAP
2016-01-14 17:16:42,796 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791802654_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/50A3138F1DCE43568600C939AA509404/D2F91A1AF5204329886E5EFA223DEC90/.tez/application_1452791802654_0001/localmode-log-dir/dag_1452791802654_0001_1.dot
2016-01-14 17:16:42,796 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/50A3138F1DCE43568600C939AA509404/D2F91A1AF5204329886E5EFA223DEC90/.tez/application_1452791802654_0001/localmode-log-dir/dag_1452791802654_0001_1-tez-dag.pb.txt
2016-01-14 17:16:42,916 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791802654_0001_1
2016-01-14 17:16:42,916 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:42,916 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [50A3138F1DCE43568600C939AA509404/D2F91A1AF5204329886E5EFA223DEC90] every/(1/1) ...emblyplatform/0_ARGS_SWAP
2016-01-14 17:16:42 Running Dag: dag_1452791802654_0001_1
2016-01-14 17:16:42,927 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791802654_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791802654_0001_1, submitTime=1452791802795
2016-01-14 17:16:42,927 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:42,927 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex A4ED25A533004799B8C410D0148233A0
2016-01-14 17:16:42,928 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:42,928 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791802654_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791802654_0001_1, initTime=1452791802927
2016-01-14 17:16:42,928 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791802654_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:42,928 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:42,928 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791802654_0001_1][Event:DAG_STARTED]: dagID=dag_1452791802654_0001_1, startTime=1452791802928
2016-01-14 17:16:42,928 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:42,928 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:42,929 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791802654_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:42,929 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: A9B9FD12DA4E47D6BCD261E10BD11733 : {F45E0EBB3B5B465A99CCEA29DF2CBA40={InputName=F45E0EBB3B5B465A99CCEA29DF2CBA40}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:42,929 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: F45E0EBB3B5B465A99CCEA29DF2CBA40, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:42,929 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791802654_0001_1_00 [A9B9FD12DA4E47D6BCD261E10BD11733]
2016-01-14 17:16:42,929 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791802654_0001_1_00 [A9B9FD12DA4E47D6BCD261E10BD11733]
2016-01-14 17:16:42,929 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791802654_0001_1_00 [A9B9FD12DA4E47D6BCD261E10BD11733]
2016-01-14 17:16:42,929 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791802654_0001_1_00 [A9B9FD12DA4E47D6BCD261E10BD11733]. Starting root input initializers: 1
2016-01-14 17:16:42,930 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791802654_0001_1_00 [A9B9FD12DA4E47D6BCD261E10BD11733] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:42,930 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791802654_0001_1_01 [A4ED25A533004799B8C410D0148233A0]
2016-01-14 17:16:42,930 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: F45E0EBB3B5B465A99CCEA29DF2CBA40 on vertex vertex_1452791802654_0001_1_00 [A9B9FD12DA4E47D6BCD261E10BD11733]
2016-01-14 17:16:42,931 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...emblyplatform/0_ARGS_SWAP, applicationId=application_1452791802654_0001, dagName=[50A3138F1DCE43568600C939AA509404/D2F91A1AF5204329886E5EFA223DEC90] every/(1/1) ...emblyplatform/0_ARGS_SWAP
2016-01-14 17:16:42,932 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:42,932 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791802654_0001_1_01 [A4ED25A533004799B8C410D0148233A0]
2016-01-14 17:16:42,932 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:42,933 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791802654_0001_1_01 [A4ED25A533004799B8C410D0148233A0]
2016-01-14 17:16:42,933 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791802654_0001_1_01 [A4ED25A533004799B8C410D0148233A0]
2016-01-14 17:16:42,933 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=B51992C56D774CC3B7ECD3D593567F02, vertexId=vertex_1452791802654_0001_1_01 [A4ED25A533004799B8C410D0148233A0], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:42,933 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input F45E0EBB3B5B465A99CCEA29DF2CBA40 asking for 1 tasks. Headroom: 683 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:42,933 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=B51992C56D774CC3B7ECD3D593567F02, vertexId=vertex_1452791802654_0001_1_01 [A4ED25A533004799B8C410D0148233A0]
2016-01-14 17:16:42,933 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:42,933 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:42,935 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:42,935 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: F45E0EBB3B5B465A99CCEA29DF2CBA40 on vertex vertex_1452791802654_0001_1_00 [A9B9FD12DA4E47D6BCD261E10BD11733]
2016-01-14 17:16:42,935 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=B51992C56D774CC3B7ECD3D593567F02, vertexName=A4ED25A533004799B8C410D0148233A0, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:42,935 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=B51992C56D774CC3B7ECD3D593567F02, vertexId=vertex_1452791802654_0001_1_01 [A4ED25A533004799B8C410D0148233A0]
2016-01-14 17:16:42,936 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791802654_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A4ED25A533004799B8C410D0148233A0, vertexId=vertex_1452791802654_0001_1_01, initRequestedTime=1452791802930, initedTime=1452791802936, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:42,936 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:42,936 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791802654_0001_1_01 [A4ED25A533004799B8C410D0148233A0] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:42,937 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791802654_0001_1_00 [A9B9FD12DA4E47D6BCD261E10BD11733] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:42,937 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {F45E0EBB3B5B465A99CCEA29DF2CBA40=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:42,937 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791802654_0001_1_00 [A9B9FD12DA4E47D6BCD261E10BD11733] parallelism set to 1
2016-01-14 17:16:42,937 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791802654_0001_1_00 [A9B9FD12DA4E47D6BCD261E10BD11733] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:42,938 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791802654_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791802654_0001_1_00, eventCount=1
2016-01-14 17:16:42,938 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:42,938 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791802654_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A9B9FD12DA4E47D6BCD261E10BD11733, vertexId=vertex_1452791802654_0001_1_00, initRequestedTime=1452791802929, initedTime=1452791802938, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:42,938 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791802654_0001_1_00 [A9B9FD12DA4E47D6BCD261E10BD11733] with distanceFromRoot: 0
2016-01-14 17:16:42,938 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791802654_0001_1_00 [A9B9FD12DA4E47D6BCD261E10BD11733] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:42,939 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in A9B9FD12DA4E47D6BCD261E10BD11733
2016-01-14 17:16:42,939 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791802654_0001_1_00 [A9B9FD12DA4E47D6BCD261E10BD11733]
2016-01-14 17:16:42,939 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:42,939 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791802654_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791802654_0001_1_00, startRequestedTime=1452791802938, startedTime=1452791802939
2016-01-14 17:16:42,939 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791802654_0001_1_00 [A9B9FD12DA4E47D6BCD261E10BD11733] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:42,940 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:42,940 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791802654_0001_1][Event:TASK_STARTED]: vertexName=A9B9FD12DA4E47D6BCD261E10BD11733, taskId=task_1452791802654_0001_1_00_000000, scheduledTime=1452791802940, launchTime=1452791802940
2016-01-14 17:16:42,940 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791802654_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:42,940 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791802654_0001_1_00 for vertex: vertex_1452791802654_0001_1_01 [A4ED25A533004799B8C410D0148233A0] numStartedSources: 1 numSources: 1
2016-01-14 17:16:42,940 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791802654_0001_1_01 [A4ED25A533004799B8C410D0148233A0] with distanceFromRoot: 1
2016-01-14 17:16:42,941 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791802654_0001_1_00 [A9B9FD12DA4E47D6BCD261E10BD11733]
2016-01-14 17:16:42,941 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: A9B9FD12DA4E47D6BCD261E10BD11733 in vertex: A4ED25A533004799B8C410D0148233A0
2016-01-14 17:16:42,941 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: A4ED25A533004799B8C410D0148233A0 with 1 source tasks and 1 pending tasks
2016-01-14 17:16:42,941 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:42,941 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791802654_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791802654_0001_1_01, startRequestedTime=1452791802941, startedTime=1452791802941
2016-01-14 17:16:42,942 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791802654_0001_1_01 [A4ED25A533004799B8C410D0148233A0] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:42,942 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [50A3138F1DCE43568600C939AA509404/D2F91A1AF5204329886E5EFA223DEC90] every/(1/1) ...emblyplatform/0_ARGS_SWAP, VertexName: A9B9FD12DA4E47D6BCD261E10BD11733, VertexParallelism: 1, TaskAttemptID:attempt_1452791802654_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F45E0EBB3B5B465A99CCEA29DF2CBA40, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A4ED25A533004799B8C410D0148233A0, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:42,942 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791802654_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:42,942 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:42,943 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:42,943 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:42,943 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:42,944 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791802654_0001_00_000001
2016-01-14 17:16:42,944 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791802654_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:42,945 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791802654_0001_00_000001
2016-01-14 17:16:42,945 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:42,946 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:42,946 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791802654_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:42,946 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791802654_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791802654_0001_00_000001, launchTime=1452791802946
2016-01-14 17:16:42,947 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791802654_0001_00_000001
2016-01-14 17:16:42,947 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791802654_0001_1_00_000000_0] to container: [container_1452791802654_0001_00_000001]
2016-01-14 17:16:42,947 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791802654_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:42,947 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791802654_0001_00_000001 given task: attempt_1452791802654_0001_1_00_000000_0
2016-01-14 17:16:42,948 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791802654_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791802654_0001_1_00_000000_0
2016-01-14 17:16:42 Starting to run new task attempt: attempt_1452791802654_0001_1_00_000000_0
2016-01-14 17:16:42,948 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791802654_0001_1_00_000000_0] started. Is using containerId: [container_1452791802654_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:42,948 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791802654_0001_1_00_000000_0
2016-01-14 17:16:42,948 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:42,948 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791802654_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A9B9FD12DA4E47D6BCD261E10BD11733, taskAttemptId=attempt_1452791802654_0001_1_00_000000_0, startTime=1452791802948, containerId=container_1452791802654_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791802654_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:42,949 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791802654_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:42,949 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791802654_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:42,948 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:42,948 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:42,949 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:42,949 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:42,949 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:42,949 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:42,950 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [50A3138F1DCE43568600C939AA509404/D2F91A1AF5204329886E5EFA223DEC90] every/(1/1) ...emblyplatform/0_ARGS_SWAP, VertexName: A9B9FD12DA4E47D6BCD261E10BD11733, VertexParallelism: 1, TaskAttemptID:attempt_1452791802654_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F45E0EBB3B5B465A99CCEA29DF2CBA40, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A4ED25A533004799B8C410D0148233A0, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:42,950 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=716701696
2016-01-14 17:16:42,950 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791802654_0001_1_00_000000_0
2016-01-14 17:16:42,950 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791802654_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:42,950 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:42,951 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:42,952 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:42,952 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F45E0EBB3B5B465A99CCEA29DF2CBA40, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:42,952 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:42,952 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F45E0EBB3B5B465A99CCEA29DF2CBA40
2016-01-14 17:16:42,952 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: F45E0EBB3B5B465A99CCEA29DF2CBA40
2016-01-14 17:16:42,954 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:42,954 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:42,954 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: F45E0EBB3B5B465A99CCEA29DF2CBA40
2016-01-14 17:16:42,954 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F45E0EBB3B5B465A99CCEA29DF2CBA40
2016-01-14 17:16:42,956 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A4ED25A533004799B8C410D0148233A0, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:42,956 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:42,956 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A4ED25A533004799B8C410D0148233A0
2016-01-14 17:16:42,958 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791802654_0001, with dag id: dag_1452791802654_0001_1
2016-01-14 17:16:42,958 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:42,958 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A9B9FD12DA4E47D6BCD261E10BD11733, ordinal: 0
2016-01-14 17:16:42,959 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A9B9FD12DA4E47D6BCD261E10BD11733, mem on start (mb), free: 629, total: 683, max: 683
2016-01-14 17:16:42,959 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:42,959 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:42,959 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:42,958 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: A4ED25A533004799B8C410D0148233A0
2016-01-14 17:16:42,959 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A4ED25A533004799B8C410D0148233A0
2016-01-14 17:16:42,959 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:42,960 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:42,960 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:42,960 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:42,960 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:42,960 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:42,960 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 716701696, TotalAvailable: 480190136, TotalRequested/TotalJVMHeap:0.15
2016-01-14 17:16:42,961 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F45E0EBB3B5B465A99CCEA29DF2CBA40, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:42,961 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A4ED25A533004799B8C410D0148233A0, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:42,961 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:42,961 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F45E0EBB3B5B465A99CCEA29DF2CBA40 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:42,961 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:42,962 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:42,961 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F45E0EBB3B5B465A99CCEA29DF2CBA40
2016-01-14 17:16:42,962 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:42,962 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F45E0EBB3B5B465A99CCEA29DF2CBA40
2016-01-14 17:16:42,963 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:42,963 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791802654_0001_1_00_000000_0
2016-01-14 17:16:42,964 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:42,965 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:42,965 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: F45E0EBB3B5B465A99CCEA29DF2CBA40
2016-01-14 17:16:42,965 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: DD118C598AF74257AF46450B4F87C2D4
2016-01-14 17:16:42,966 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] DD118C598AF74257AF46450B4F87C2D4
2016-01-14 17:16:42,966 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@3fbcd2c
2016-01-14 17:16:42,966 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@85b05bf; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@fd642e2; comparator=cascading.tuple.hadoop.util.TupleComparator@3bc10b8d; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:42,966 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:42,966 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:42,966 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:42,967 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@3fbcd2c
2016-01-14 17:16:42,967 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:42,967 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:42,979 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:42,979 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:42,979 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:42,979 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:42,979 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:42,979 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:42,980 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:42,980 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A9B9FD12DA4E47D6BCD261E10BD11733, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:42,982 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791802654_0001_1_00_000000_0
2016-01-14 17:16:42,982 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A9B9FD12DA4E47D6BCD261E10BD11733, mem on close (mb), free: 527, total: 683, max: 683
2016-01-14 17:16:42,982 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:42,982 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:42,982 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:42,982 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:42,984 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:42 Completed running task attempt: attempt_1452791802654_0001_1_00_000000_0
2016-01-14 17:16:42,987 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791802654_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:42,987 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=716701696, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791802980, Process_Duration=2, Process_End_Time=1452791802982, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=2][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:42,987 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791802654_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791802654_0001_1_00, eventCount=1
2016-01-14 17:16:42,988 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:42,988 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:42,988 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791802654_0001_00_000001
2016-01-14 17:16:42,988 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791802654_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A9B9FD12DA4E47D6BCD261E10BD11733, taskAttemptId=attempt_1452791802654_0001_1_00_000000_0, startTime=1452791802948, finishTime=1452791802988, timeTaken=40, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=716701696, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791802980, Process_Duration=2, Process_End_Time=1452791802982, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:42,988 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791802654_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:42,989 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791802654_0001_1_00_000000_0
2016-01-14 17:16:42,989 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791802654_0001_1][Event:TASK_FINISHED]: vertexName=A9B9FD12DA4E47D6BCD261E10BD11733, taskId=task_1452791802654_0001_1_00_000000, startTime=1452791802948, finishTime=1452791802989, timeTaken=41, status=SUCCEEDED, successfulAttemptID=attempt_1452791802654_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=716701696, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791802980, Process_Duration=2, Process_End_Time=1452791802982, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:42,989 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791802654_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:42,990 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791802654_0001_1_00 [A9B9FD12DA4E47D6BCD261E10BD11733] : 1
2016-01-14 17:16:42,990 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791802654_0001_1_00 [A9B9FD12DA4E47D6BCD261E10BD11733]
2016-01-14 17:16:42,990 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:42,990 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791802654_0001_1][Event:VERTEX_FINISHED]: vertexName=A9B9FD12DA4E47D6BCD261E10BD11733, vertexId=vertex_1452791802654_0001_1_00, initRequestedTime=1452791802929, initedTime=1452791802938, startRequestedTime=1452791802938, startedTime=1452791802939, finishTime=1452791802990, timeTaken=51, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=716701696, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791802980, Process_Duration=2, Process_End_Time=1452791802982, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791802948, firstTasksToStart=[ task_1452791802654_0001_1_00_000000 ], lastTaskFinishTime=1452791802988, lastTasksToFinish=[ task_1452791802654_0001_1_00_000000 ], minTaskDuration=40, maxTaskDuration=40, avgTaskDuration=40.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791802654_0001_1_00_000000 ], longestDurationTasks=[ task_1452791802654_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:42,990 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791802654_0001_1_00 [A9B9FD12DA4E47D6BCD261E10BD11733] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:42,990 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791802654_0001_1_01 [A4ED25A533004799B8C410D0148233A0] attempt: attempt_1452791802654_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:42,990 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: A4ED25A533004799B8C410D0148233A0
2016-01-14 17:16:42,991 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791802654_0001_1_01 [A4ED25A533004799B8C410D0148233A0]
2016-01-14 17:16:42,991 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791802654_0001_1_00 [A9B9FD12DA4E47D6BCD261E10BD11733] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:42,991 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:42,991 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791802654_0001_1][Event:TASK_STARTED]: vertexName=A4ED25A533004799B8C410D0148233A0, taskId=task_1452791802654_0001_1_01_000000, scheduledTime=1452791802991, launchTime=1452791802991
2016-01-14 17:16:42,991 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791802654_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:42,991 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791802654_0001_1_01 [A4ED25A533004799B8C410D0148233A0]
2016-01-14 17:16:42,991 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:42,991 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:42,992 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:42,991 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [50A3138F1DCE43568600C939AA509404/D2F91A1AF5204329886E5EFA223DEC90] every/(1/1) ...emblyplatform/0_ARGS_SWAP, VertexName: A4ED25A533004799B8C410D0148233A0, VertexParallelism: 1, TaskAttemptID:attempt_1452791802654_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A9B9FD12DA4E47D6BCD261E10BD11733, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=B51992C56D774CC3B7ECD3D593567F02, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:42,993 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791802654_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:42,993 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791802654_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:42,993 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791802654_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791802654_0001_00_000001, stoppedTime=1452791802993, exitStatus=0
2016-01-14 17:16:42,993 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:42,993 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791802654_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:42,993 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:42,993 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791802654_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:42,994 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791802654_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:42,994 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791802654_0001_00_000002
2016-01-14 17:16:42,994 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791802654_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:42,996 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791802654_0001_00_000002
2016-01-14 17:16:42,996 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:42,996 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791802654_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:42,997 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:42,997 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791802654_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791802654_0001_00_000002, launchTime=1452791802996
2016-01-14 17:16:42,997 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791802654_0001_00_000002
2016-01-14 17:16:42,998 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791802654_0001_1_01_000000_0] to container: [container_1452791802654_0001_00_000002]
2016-01-14 17:16:42,998 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791802654_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:42,998 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791802654_0001_1_01_000000_0] started. Is using containerId: [container_1452791802654_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:42,998 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791802654_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A4ED25A533004799B8C410D0148233A0, taskAttemptId=attempt_1452791802654_0001_1_01_000000_0, startTime=1452791802998, containerId=container_1452791802654_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791802654_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:42,998 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791802654_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:42,998 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:42,998 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791802654_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:42,999 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791802654_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:42,999 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791802654_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:42,999 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791802654_0001_00_000001, Exiting
2016-01-14 17:16:42,999 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791802654_0001_00_000001
2016-01-14 17:16:42,999 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791802654_0001_00_000001
2016-01-14 17:16:42,999 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791802654_0001_00_000002 given task: attempt_1452791802654_0001_1_01_000000_0
2016-01-14 17:16:43,000 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791802654_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791802654_0001_1_01_000000_0
2016-01-14 17:16:43 Starting to run new task attempt: attempt_1452791802654_0001_1_01_000000_0
2016-01-14 17:16:43,000 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791802654_0001_1_01_000000_0
2016-01-14 17:16:43,000 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:43,000 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:43,000 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:43,000 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:43,001 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:43,001 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:43,001 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [50A3138F1DCE43568600C939AA509404/D2F91A1AF5204329886E5EFA223DEC90] every/(1/1) ...emblyplatform/0_ARGS_SWAP, VertexName: A4ED25A533004799B8C410D0148233A0, VertexParallelism: 1, TaskAttemptID:attempt_1452791802654_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A9B9FD12DA4E47D6BCD261E10BD11733, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=B51992C56D774CC3B7ECD3D593567F02, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:43,001 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=716701696
2016-01-14 17:16:43,001 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791802654_0001_1_01_000000_0
2016-01-14 17:16:43,001 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:43,001 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:43,002 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:43,003 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=A9B9FD12DA4E47D6BCD261E10BD11733, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:43,003 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:43,003 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: A9B9FD12DA4E47D6BCD261E10BD11733
2016-01-14 17:16:43,004 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791802654_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:43,004 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791802654_0001_00_000001 completed successfully
2016-01-14 17:16:43,005 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:43,005 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 645031488, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:43,005 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:43,005 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 645031488, type: INPUT, componentVertexName: A9B9FD12DA4E47D6BCD261E10BD11733
2016-01-14 17:16:43,005 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: A9B9FD12DA4E47D6BCD261E10BD11733
2016-01-14 17:16:43,006 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791802654_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:43,006 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791802654_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:43,007 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=B51992C56D774CC3B7ECD3D593567F02, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:43,008 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:43,008 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: B51992C56D774CC3B7ECD3D593567F02
2016-01-14 17:16:43,008 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:43,010 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: B51992C56D774CC3B7ECD3D593567F02
2016-01-14 17:16:43,010 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A4ED25A533004799B8C410D0148233A0, ordinal: 1
2016-01-14 17:16:43,012 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A4ED25A533004799B8C410D0148233A0, mem on start (mb), free: 521, total: 683, max: 683
2016-01-14 17:16:43,012 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:43,013 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:43,013 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:43,018 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:43,018 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: B51992C56D774CC3B7ECD3D593567F02
2016-01-14 17:16:43,018 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:43,018 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:43,018 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:43,018 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:43,019 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:43,019 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:43,019 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 645031488, TotalRequestedScaled: 5.954136812307693E8, TotalJVMHeap: 716701696, TotalAvailable: 480190136, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:43,019 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, A9B9FD12DA4E47D6BCD261E10BD11733, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=645031488, allocated=480190136
2016-01-14 17:16:43,019 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, B51992C56D774CC3B7ECD3D593567F02, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:43,019 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:43,019 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: A9B9FD12DA4E47D6BCD261E10BD11733 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:43,019 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:43,019 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:43,019 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: A9B9FD12DA4E47D6BCD261E10BD11733
2016-01-14 17:16:43,020 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:43,020 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:43,021 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: A9B9FD12DA4E47D6BCD261E10BD11733 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:43,021 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=645031488, postMergeMem=0, RuntimeTotalAvailable=480190136. Updated to: ShuffleMem=480190136, postMergeMem: 0
2016-01-14 17:16:43,021 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=480190136, maxSingleShuffleLimit=120047536, mergeThreshold=432171104, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:43,021 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:43,023 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: A9B9FD12DA4E47D6BCD261E10BD11733
2016-01-14 17:16:43,023 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:43,023 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791802654_0001_1_01_000000_0
2016-01-14 17:16:43,024 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791802654_0001_1_00_000000_0_10225, runDuration: 29881, ]
2016-01-14 17:16:43,026 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:43,026 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: DD118C598AF74257AF46450B4F87C2D4
2016-01-14 17:16:43,026 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791802654_0001_1_00_000000_0_10225] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791803026, TimeTaken=0, Rate=0.00 MB/s
2016-01-14 17:16:43,026 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_ARGS_SWAP"], id: B51992C56D774CC3B7ECD3D593567F02
2016-01-14 17:16:43,027 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_ARGS_SWAP"]
2016-01-14 17:16:43,027 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] DD118C598AF74257AF46450B4F87C2D4, for 1 inputs
2016-01-14 17:16:43,026 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : A9B9FD12DA4E47D6BCD261E10BD11733
2016-01-14 17:16:43,027 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:43,028 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [A9B9FD12DA4E47D6BCD261E10BD11733] #38 in 3ms
2016-01-14 17:16:43,028 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [A9B9FD12DA4E47D6BCD261E10BD11733] #38, status:false, isInterrupted:false
2016-01-14 17:16:43,028 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:43,028 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:43,029 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:43,029 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:43,029 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:43,029 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : A9B9FD12DA4E47D6BCD261E10BD11733
2016-01-14 17:16:43,029 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:43,030 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A4ED25A533004799B8C410D0148233A0, all 1 inputs ready in: 00:00:00.003
2016-01-14 17:16:43,030 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@374c260a; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@685576e2
2016-01-14 17:16:43,034 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:43,034 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:43,035 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791802654_0001_1_01_000000_0
2016-01-14 17:16:43,035 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791802654_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:43,036 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791802654_0001_1_01_000000_0
2016-01-14 17:16:43,036 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A4ED25A533004799B8C410D0148233A0, mem on close (mb), free: 518, total: 683, max: 683
2016-01-14 17:16:43,036 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: A9B9FD12DA4E47D6BCD261E10BD11733
2016-01-14 17:16:43,036 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:43,037 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:43,037 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:43,037 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:43,037 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:43 Completed running task attempt: attempt_1452791802654_0001_1_01_000000_0
2016-01-14 17:16:43,038 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791802654_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:43,039 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=183, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=716701696, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791803030, Process_Duration=6, Process_End_Time=1452791803036, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 17:16:43,039 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:43,039 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:43,040 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:43,039 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791802654_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A4ED25A533004799B8C410D0148233A0, taskAttemptId=attempt_1452791802654_0001_1_01_000000_0, startTime=1452791802998, finishTime=1452791803039, timeTaken=41, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=183, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=716701696, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791803030, Process_Duration=6, Process_End_Time=1452791803036, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:43,040 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791802654_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:43,040 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791802654_0001_00_000002
2016-01-14 17:16:43,040 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:43,040 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791802654_0001_1_01_000000_0
2016-01-14 17:16:43,041 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791802654_0001_1][Event:TASK_FINISHED]: vertexName=A4ED25A533004799B8C410D0148233A0, taskId=task_1452791802654_0001_1_01_000000, startTime=1452791802998, finishTime=1452791803040, timeTaken=42, status=SUCCEEDED, successfulAttemptID=attempt_1452791802654_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=183, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=716701696, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791803030, Process_Duration=6, Process_End_Time=1452791803036, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:43,041 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791802654_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:43,041 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791802654_0001_1_01 [A4ED25A533004799B8C410D0148233A0] : 1
2016-01-14 17:16:43,041 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791802654_0001_1_01 [A4ED25A533004799B8C410D0148233A0]
2016-01-14 17:16:43,042 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791802654_0001_1][Event:VERTEX_FINISHED]: vertexName=A4ED25A533004799B8C410D0148233A0, vertexId=vertex_1452791802654_0001_1_01, initRequestedTime=1452791802930, initedTime=1452791802936, startRequestedTime=1452791802941, startedTime=1452791802941, finishTime=1452791803041, timeTaken=100, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=183, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=716701696, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791803030, Process_Duration=6, Process_End_Time=1452791803036, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452791802998, firstTasksToStart=[ task_1452791802654_0001_1_01_000000 ], lastTaskFinishTime=1452791803039, lastTasksToFinish=[ task_1452791802654_0001_1_01_000000 ], minTaskDuration=41, maxTaskDuration=41, avgTaskDuration=41.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791802654_0001_1_01_000000 ], longestDurationTasks=[ task_1452791802654_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:43,042 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791802654_0001_1_01 [A4ED25A533004799B8C410D0148233A0] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:43,042 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791802654_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:43,042 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791802654_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791802654_0001_00_000002, stoppedTime=1452791803042, exitStatus=0
2016-01-14 17:16:43,042 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791802654_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:43,042 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791802654_0001_1_01 [A4ED25A533004799B8C410D0148233A0] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:43,042 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791802654_0001_1
2016-01-14 17:16:43,042 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791802654_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452791802654_0001_1
2016-01-14 17:16:43,042 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: B51992C56D774CC3B7ECD3D593567F02 for vertex: vertex_1452791802654_0001_1_01 [A4ED25A533004799B8C410D0148233A0]
2016-01-14 17:16:43,044 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:43,045 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:43,045 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:43,051 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452791802654_0001_1_00 [A9B9FD12DA4E47D6BCD261E10BD11733]
2016-01-14 17:16:43,051 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791802654_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 17:16:43,051 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791802654_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:43,051 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791802654_0001_00_000002, Exiting
2016-01-14 17:16:43,051 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791802654_0001_00_000002
2016-01-14 17:16:43,051 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791802654_0001_00_000002
2016-01-14 17:16:43,052 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791802654_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:43,052 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791802654_0001_00_000002 completed successfully
2016-01-14 17:16:43,053 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791802654_0001_1, queueSize=0
2016-01-14 17:16:43,053 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:43,054 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791802654_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791802654_0001_1, startTime=1452791802928, finishTime=1452791803051, timeTaken=123, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=1036, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1433403392, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905583606010, Process_Duration=8, Process_End_Time=2905583606018, Read_Duration=0, Tuples_Read=40, Tuples_Written=40, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 17:16:43,054 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791802654_0001_1 finished with state: SUCCEEDED
2016-01-14 17:16:43,054 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791802654_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:43,055 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791802654_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:16:43,055 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452791802654_0001_00_000002
2016-01-14 17:16:43 Completed Dag: dag_1452791802654_0001_1
2016-01-14 17:16:43,058 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791802654_0001_1, dagState=SUCCEEDED
2016-01-14 17:16:43,059 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791802654_0001_1_post
2016-01-14 17:16:43,059 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:43,060 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:43,060 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...emblyplatform/0_ARGS_SWAP, applicationId=application_1452791802654_0001
2016-01-14 17:16:43,062 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:43,062 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:43,062 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:43,062 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:43,062 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:43,063 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:43,063 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:43,063 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:43,063 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:43,063 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:43,063 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:43,064 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:43,064 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:43,069 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/50A3138F1DCE43568600C939AA509404/D2F91A1AF5204329886E5EFA223DEC90/.tez/application_1452791802654_0001
2016-01-14 17:16:43,069 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.431
2016-01-14 17:16:43,129 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_ARGS_SWAP
2016-01-14 17:16:43,149 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:43,149 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:43,178 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.006
2016-01-14 17:16:43,184 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-01-14 17:16:43,185 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:43,185 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:43,186 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:43,197 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:43,202 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_-1
2016-01-14 17:16:43,222 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 17:16:43,222 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 17:16:43,243 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:43,243 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:43,243 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_-1"]
2016-01-14 17:16:43,243 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:43,243 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:43,243 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:43,244 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...lyplatform/ALL_'field'_-1
2016-01-14 17:16:43,245 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:43,257 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:43,261 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:43,262 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:43,262 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:43,271 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3FB9BBF3D05540678AC3B1088B65152A/A90F0F23B2674A0390336D55E56DE6B4/.tez/application_1452791803262_0001 doesn't exist and is created
2016-01-14 17:16:43,288 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:43,288 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:43,288 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3FB9BBF3D05540678AC3B1088B65152A/A90F0F23B2674A0390336D55E56DE6B4/.tez/application_1452791803262_0001
2016-01-14 17:16:43,290 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791803262_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:43,300 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:43,300 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:43,301 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:43,301 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:43,301 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3FB9BBF3D05540678AC3B1088B65152A/A90F0F23B2674A0390336D55E56DE6B4/.tez/application_1452791803262_0001/localmode-log-dir
2016-01-14 17:16:43,301 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3FB9BBF3D05540678AC3B1088B65152A/A90F0F23B2674A0390336D55E56DE6B4/.tez/application_1452791803262_0001/localmode-log-dir/history.txt.appattempt_1452791803262_0001_000000, maxErrors=10
2016-01-14 17:16:43,301 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:43,301 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791803262_0001_000000, appSubmitTime=1452791803290, launchTime=1452791803290
2016-01-14 17:16:43,304 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:43,305 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:43,306 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:43,308 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:58097
2016-01-14 17:16:43,313 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:43,313 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:43,315 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791803262_0001_000000, startTime=1452791803315
2016-01-14 17:16:43,315 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:43,315 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:43,388 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:43,388 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:43,388 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...lyplatform/ALL_'field'_-1, applicationId=application_1452791803262_0001, dagName=[3FB9BBF3D05540678AC3B1088B65152A/A90F0F23B2674A0390336D55E56DE6B4] every/(1/1) ...lyplatform/ALL_'field'_-1
2016-01-14 17:16:43,403 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [3FB9BBF3D05540678AC3B1088B65152A/A90F0F23B2674A0390336D55E56DE6B4] every/(1/1) ...lyplatform/ALL_'field'_-1
2016-01-14 17:16:43,405 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791803262_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3FB9BBF3D05540678AC3B1088B65152A/A90F0F23B2674A0390336D55E56DE6B4/.tez/application_1452791803262_0001/localmode-log-dir/dag_1452791803262_0001_1.dot
2016-01-14 17:16:43,405 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3FB9BBF3D05540678AC3B1088B65152A/A90F0F23B2674A0390336D55E56DE6B4/.tez/application_1452791803262_0001/localmode-log-dir/dag_1452791803262_0001_1-tez-dag.pb.txt
2016-01-14 17:16:43,411 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791803262_0001_1
2016-01-14 17:16:43,411 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:43,411 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [3FB9BBF3D05540678AC3B1088B65152A/A90F0F23B2674A0390336D55E56DE6B4] every/(1/1) ...lyplatform/ALL_'field'_-1
2016-01-14 17:16:43 Running Dag: dag_1452791803262_0001_1
2016-01-14 17:16:43,423 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791803262_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791803262_0001_1, submitTime=1452791803404
2016-01-14 17:16:43,423 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:43,423 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex E5E2B7825E4C4D62AA87194E88F19494
2016-01-14 17:16:43,423 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:43,424 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791803262_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791803262_0001_1, initTime=1452791803423
2016-01-14 17:16:43,424 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:43,424 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791803262_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:43,424 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791803262_0001_1][Event:DAG_STARTED]: dagID=dag_1452791803262_0001_1, startTime=1452791803424
2016-01-14 17:16:43,424 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:43,424 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:43,424 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791803262_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:43,425 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D0276CE57BE2453A92A2C2985821152E : {51ECF853E88E4EBFB6BD20A6E85BC5D7={InputName=51ECF853E88E4EBFB6BD20A6E85BC5D7}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:43,425 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 51ECF853E88E4EBFB6BD20A6E85BC5D7, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:43,425 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791803262_0001_1_00 [D0276CE57BE2453A92A2C2985821152E]
2016-01-14 17:16:43,425 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791803262_0001_1_00 [D0276CE57BE2453A92A2C2985821152E]
2016-01-14 17:16:43,425 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791803262_0001_1_00 [D0276CE57BE2453A92A2C2985821152E]
2016-01-14 17:16:43,425 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...lyplatform/ALL_'field'_-1, applicationId=application_1452791803262_0001, dagName=[3FB9BBF3D05540678AC3B1088B65152A/A90F0F23B2674A0390336D55E56DE6B4] every/(1/1) ...lyplatform/ALL_'field'_-1
2016-01-14 17:16:43,425 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791803262_0001_1_00 [D0276CE57BE2453A92A2C2985821152E]. Starting root input initializers: 1
2016-01-14 17:16:43,428 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791803262_0001_1_00 [D0276CE57BE2453A92A2C2985821152E] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:43,428 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 51ECF853E88E4EBFB6BD20A6E85BC5D7 on vertex vertex_1452791803262_0001_1_00 [D0276CE57BE2453A92A2C2985821152E]
2016-01-14 17:16:43,428 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791803262_0001_1_01 [E5E2B7825E4C4D62AA87194E88F19494]
2016-01-14 17:16:43,431 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:43,431 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 51ECF853E88E4EBFB6BD20A6E85BC5D7 asking for 1 tasks. Headroom: 683 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:43,431 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:43,432 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791803262_0001_1_01 [E5E2B7825E4C4D62AA87194E88F19494]
2016-01-14 17:16:43,432 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791803262_0001_1_01 [E5E2B7825E4C4D62AA87194E88F19494]
2016-01-14 17:16:43,432 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791803262_0001_1_01 [E5E2B7825E4C4D62AA87194E88F19494]
2016-01-14 17:16:43,432 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=8829B94C076A4C9BA7B450B80B4D8BB4, vertexId=vertex_1452791803262_0001_1_01 [E5E2B7825E4C4D62AA87194E88F19494], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:43,432 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=8829B94C076A4C9BA7B450B80B4D8BB4, vertexId=vertex_1452791803262_0001_1_01 [E5E2B7825E4C4D62AA87194E88F19494]
2016-01-14 17:16:43,434 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=8829B94C076A4C9BA7B450B80B4D8BB4, vertexName=E5E2B7825E4C4D62AA87194E88F19494, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:43,434 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=8829B94C076A4C9BA7B450B80B4D8BB4, vertexId=vertex_1452791803262_0001_1_01 [E5E2B7825E4C4D62AA87194E88F19494]
2016-01-14 17:16:43,435 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:43,435 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791803262_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E5E2B7825E4C4D62AA87194E88F19494, vertexId=vertex_1452791803262_0001_1_01, initRequestedTime=1452791803428, initedTime=1452791803435, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:43,435 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791803262_0001_1_01 [E5E2B7825E4C4D62AA87194E88F19494] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:43,432 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:43,436 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:43,437 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:43,437 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 51ECF853E88E4EBFB6BD20A6E85BC5D7 on vertex vertex_1452791803262_0001_1_00 [D0276CE57BE2453A92A2C2985821152E]
2016-01-14 17:16:43,438 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791803262_0001_1_00 [D0276CE57BE2453A92A2C2985821152E] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:43,438 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {51ECF853E88E4EBFB6BD20A6E85BC5D7=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:43,438 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791803262_0001_1_00 [D0276CE57BE2453A92A2C2985821152E] parallelism set to 1
2016-01-14 17:16:43,438 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791803262_0001_1_00 [D0276CE57BE2453A92A2C2985821152E] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:43,438 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791803262_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791803262_0001_1_00, eventCount=1
2016-01-14 17:16:43,439 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:43,439 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791803262_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D0276CE57BE2453A92A2C2985821152E, vertexId=vertex_1452791803262_0001_1_00, initRequestedTime=1452791803425, initedTime=1452791803439, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:43,439 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791803262_0001_1_00 [D0276CE57BE2453A92A2C2985821152E] with distanceFromRoot: 0
2016-01-14 17:16:43,439 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791803262_0001_1_00 [D0276CE57BE2453A92A2C2985821152E] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:43,440 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D0276CE57BE2453A92A2C2985821152E
2016-01-14 17:16:43,440 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791803262_0001_1_00 [D0276CE57BE2453A92A2C2985821152E]
2016-01-14 17:16:43,440 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:43,440 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791803262_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791803262_0001_1_00, startRequestedTime=1452791803440, startedTime=1452791803440
2016-01-14 17:16:43,441 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791803262_0001_1_00 [D0276CE57BE2453A92A2C2985821152E] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:43,441 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:43,441 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791803262_0001_1][Event:TASK_STARTED]: vertexName=D0276CE57BE2453A92A2C2985821152E, taskId=task_1452791803262_0001_1_00_000000, scheduledTime=1452791803441, launchTime=1452791803441
2016-01-14 17:16:43,441 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791803262_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:43,442 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791803262_0001_1_00 for vertex: vertex_1452791803262_0001_1_01 [E5E2B7825E4C4D62AA87194E88F19494] numStartedSources: 1 numSources: 1
2016-01-14 17:16:43,442 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791803262_0001_1_01 [E5E2B7825E4C4D62AA87194E88F19494] with distanceFromRoot: 1
2016-01-14 17:16:43,442 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791803262_0001_1_00 [D0276CE57BE2453A92A2C2985821152E]
2016-01-14 17:16:43,442 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: D0276CE57BE2453A92A2C2985821152E in vertex: E5E2B7825E4C4D62AA87194E88F19494
2016-01-14 17:16:43,442 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: E5E2B7825E4C4D62AA87194E88F19494 with 1 source tasks and 1 pending tasks
2016-01-14 17:16:43,443 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:43,443 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791803262_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791803262_0001_1_01, startRequestedTime=1452791803442, startedTime=1452791803442
2016-01-14 17:16:43,443 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791803262_0001_1_01 [E5E2B7825E4C4D62AA87194E88F19494] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:43,443 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [3FB9BBF3D05540678AC3B1088B65152A/A90F0F23B2674A0390336D55E56DE6B4] every/(1/1) ...lyplatform/ALL_'field'_-1, VertexName: D0276CE57BE2453A92A2C2985821152E, VertexParallelism: 1, TaskAttemptID:attempt_1452791803262_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=51ECF853E88E4EBFB6BD20A6E85BC5D7, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E5E2B7825E4C4D62AA87194E88F19494, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:43,443 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791803262_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:43,444 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:43,441 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791803262_0001, with dag id: dag_1452791803262_0001_1
2016-01-14 17:16:43,445 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:43,445 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:43,445 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:43,445 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791803262_0001_00_000001
2016-01-14 17:16:43,446 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791803262_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:43,447 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791803262_0001_00_000001
2016-01-14 17:16:43,447 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:43,448 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791803262_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:43,449 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:43,449 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791803262_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791803262_0001_00_000001, launchTime=1452791803448
2016-01-14 17:16:43,450 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791803262_0001_00_000001
2016-01-14 17:16:43,450 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791803262_0001_1_00_000000_0] to container: [container_1452791803262_0001_00_000001]
2016-01-14 17:16:43,451 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791803262_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:43,451 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791803262_0001_1_00_000000_0] started. Is using containerId: [container_1452791803262_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:43,452 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791803262_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D0276CE57BE2453A92A2C2985821152E, taskAttemptId=attempt_1452791803262_0001_1_00_000000_0, startTime=1452791803451, containerId=container_1452791803262_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791803262_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:43,452 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791803262_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:43,452 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791803262_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:43,452 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:43,453 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791803262_0001_00_000001 given task: attempt_1452791803262_0001_1_00_000000_0
2016-01-14 17:16:43,454 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791803262_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791803262_0001_1_00_000000_0
2016-01-14 17:16:43 Starting to run new task attempt: attempt_1452791803262_0001_1_00_000000_0
2016-01-14 17:16:43,455 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791803262_0001_1_00_000000_0
2016-01-14 17:16:43,455 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:43,455 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:43,455 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:43,455 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:43,455 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:43,455 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:43,455 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [3FB9BBF3D05540678AC3B1088B65152A/A90F0F23B2674A0390336D55E56DE6B4] every/(1/1) ...lyplatform/ALL_'field'_-1, VertexName: D0276CE57BE2453A92A2C2985821152E, VertexParallelism: 1, TaskAttemptID:attempt_1452791803262_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=51ECF853E88E4EBFB6BD20A6E85BC5D7, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E5E2B7825E4C4D62AA87194E88F19494, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:43,455 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=716177408
2016-01-14 17:16:43,458 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791803262_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:43,458 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791803262_0001_1_00_000000_0
2016-01-14 17:16:43,459 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:43,459 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:43,460 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=51ECF853E88E4EBFB6BD20A6E85BC5D7, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:43,460 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:43,460 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 51ECF853E88E4EBFB6BD20A6E85BC5D7
2016-01-14 17:16:43,460 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 51ECF853E88E4EBFB6BD20A6E85BC5D7
2016-01-14 17:16:43,462 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:43,462 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 51ECF853E88E4EBFB6BD20A6E85BC5D7
2016-01-14 17:16:43,463 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 51ECF853E88E4EBFB6BD20A6E85BC5D7
2016-01-14 17:16:43,463 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:43,466 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:43,468 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=E5E2B7825E4C4D62AA87194E88F19494, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:43,469 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:43,469 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: E5E2B7825E4C4D62AA87194E88F19494
2016-01-14 17:16:43,471 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:43,471 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: E5E2B7825E4C4D62AA87194E88F19494
2016-01-14 17:16:43,471 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: E5E2B7825E4C4D62AA87194E88F19494
2016-01-14 17:16:43,471 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D0276CE57BE2453A92A2C2985821152E, ordinal: 0
2016-01-14 17:16:43,471 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D0276CE57BE2453A92A2C2985821152E, mem on start (mb), free: 615, total: 683, max: 683
2016-01-14 17:16:43,471 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:43,471 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:43,472 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:43,472 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:43,472 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:43,472 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:43,472 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:43,472 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:43,472 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:43,472 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 716177408, TotalAvailable: 479838863, TotalRequested/TotalJVMHeap:0.15
2016-01-14 17:16:43,472 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 51ECF853E88E4EBFB6BD20A6E85BC5D7, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:43,472 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, E5E2B7825E4C4D62AA87194E88F19494, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:43,473 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:43,473 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 51ECF853E88E4EBFB6BD20A6E85BC5D7 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:43,473 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:43,473 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 51ECF853E88E4EBFB6BD20A6E85BC5D7
2016-01-14 17:16:43,474 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:43,474 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 51ECF853E88E4EBFB6BD20A6E85BC5D7
2016-01-14 17:16:43,475 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:43,475 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:43,475 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791803262_0001_1_00_000000_0
2016-01-14 17:16:43,478 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:43,479 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 51ECF853E88E4EBFB6BD20A6E85BC5D7
2016-01-14 17:16:43,479 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 6E06FAEB221F4C61BE325799909811B6
2016-01-14 17:16:43,479 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 6E06FAEB221F4C61BE325799909811B6
2016-01-14 17:16:43,479 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@645f746a; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@36d40ea1; comparator=cascading.tuple.hadoop.util.TupleComparator@1c0c8826; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:43,480 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:43,480 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:43,480 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:43,481 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:43,481 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@5a74961d
2016-01-14 17:16:43,482 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@5a74961d
2016-01-14 17:16:43,482 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:43,482 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:43,493 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:43,493 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:43,493 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:43,493 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:43,493 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:43,494 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:43,495 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:43,495 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D0276CE57BE2453A92A2C2985821152E, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:43,498 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791803262_0001_1_00_000000_0
2016-01-14 17:16:43,498 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D0276CE57BE2453A92A2C2985821152E, mem on close (mb), free: 514, total: 683, max: 683
2016-01-14 17:16:43,498 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:43,498 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:43,498 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:43,498 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:43,501 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:43 Completed running task attempt: attempt_1452791803262_0001_1_00_000000_0
2016-01-14 17:16:43,503 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791803262_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:43,504 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=716177408, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791803495, Process_Duration=3, Process_End_Time=1452791803498, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:43,504 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791803262_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791803262_0001_1_00, eventCount=1
2016-01-14 17:16:43,504 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:43,504 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:43,504 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791803262_0001_00_000001
2016-01-14 17:16:43,504 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791803262_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D0276CE57BE2453A92A2C2985821152E, taskAttemptId=attempt_1452791803262_0001_1_00_000000_0, startTime=1452791803451, finishTime=1452791803504, timeTaken=53, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=716177408, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791803495, Process_Duration=3, Process_End_Time=1452791803498, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:43,504 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:43,504 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791803262_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:43,505 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:43,505 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791803262_0001_1_00_000000_0
2016-01-14 17:16:43,505 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:43,505 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791803262_0001_1][Event:TASK_FINISHED]: vertexName=D0276CE57BE2453A92A2C2985821152E, taskId=task_1452791803262_0001_1_00_000000, startTime=1452791803451, finishTime=1452791803505, timeTaken=54, status=SUCCEEDED, successfulAttemptID=attempt_1452791803262_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=716177408, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791803495, Process_Duration=3, Process_End_Time=1452791803498, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:43,506 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791803262_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:43,506 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791803262_0001_1_00 [D0276CE57BE2453A92A2C2985821152E] : 1
2016-01-14 17:16:43,506 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791803262_0001_1_00 [D0276CE57BE2453A92A2C2985821152E]
2016-01-14 17:16:43,506 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:43,506 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791803262_0001_1][Event:VERTEX_FINISHED]: vertexName=D0276CE57BE2453A92A2C2985821152E, vertexId=vertex_1452791803262_0001_1_00, initRequestedTime=1452791803425, initedTime=1452791803439, startRequestedTime=1452791803440, startedTime=1452791803440, finishTime=1452791803506, timeTaken=66, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=716177408, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791803495, Process_Duration=3, Process_End_Time=1452791803498, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791803451, firstTasksToStart=[ task_1452791803262_0001_1_00_000000 ], lastTaskFinishTime=1452791803504, lastTasksToFinish=[ task_1452791803262_0001_1_00_000000 ], minTaskDuration=53, maxTaskDuration=53, avgTaskDuration=53.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791803262_0001_1_00_000000 ], longestDurationTasks=[ task_1452791803262_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:43,507 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791803262_0001_1_00 [D0276CE57BE2453A92A2C2985821152E] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:43,507 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791803262_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:43,507 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791803262_0001_1_01 [E5E2B7825E4C4D62AA87194E88F19494] attempt: attempt_1452791803262_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:43,507 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: E5E2B7825E4C4D62AA87194E88F19494
2016-01-14 17:16:43,507 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791803262_0001_1_01 [E5E2B7825E4C4D62AA87194E88F19494]
2016-01-14 17:16:43,507 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791803262_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791803262_0001_00_000001, stoppedTime=1452791803507, exitStatus=0
2016-01-14 17:16:43,507 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:43,507 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791803262_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:43,508 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791803262_0001_1_00 [D0276CE57BE2453A92A2C2985821152E] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:43,508 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791803262_0001_1][Event:TASK_STARTED]: vertexName=E5E2B7825E4C4D62AA87194E88F19494, taskId=task_1452791803262_0001_1_01_000000, scheduledTime=1452791803508, launchTime=1452791803508
2016-01-14 17:16:43,508 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791803262_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:43,508 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791803262_0001_1_01 [E5E2B7825E4C4D62AA87194E88F19494]
2016-01-14 17:16:43,508 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:43,508 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791803262_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:43,508 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [3FB9BBF3D05540678AC3B1088B65152A/A90F0F23B2674A0390336D55E56DE6B4] every/(1/1) ...lyplatform/ALL_'field'_-1, VertexName: E5E2B7825E4C4D62AA87194E88F19494, VertexParallelism: 1, TaskAttemptID:attempt_1452791803262_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D0276CE57BE2453A92A2C2985821152E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=8829B94C076A4C9BA7B450B80B4D8BB4, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:43,508 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791803262_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:43,509 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791803262_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:43,509 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:43,509 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791803262_0001_00_000002
2016-01-14 17:16:43,509 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791803262_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:43,511 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791803262_0001_00_000002
2016-01-14 17:16:43,511 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:43,512 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791803262_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:43,512 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:43,512 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791803262_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791803262_0001_00_000002, launchTime=1452791803511
2016-01-14 17:16:43,512 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791803262_0001_00_000002
2016-01-14 17:16:43,513 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791803262_0001_1_01_000000_0] to container: [container_1452791803262_0001_00_000002]
2016-01-14 17:16:43,513 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791803262_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:43,513 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791803262_0001_00_000002 given task: attempt_1452791803262_0001_1_01_000000_0
2016-01-14 17:16:43,513 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791803262_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791803262_0001_1_01_000000_0
2016-01-14 17:16:43,513 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791803262_0001_1_01_000000_0] started. Is using containerId: [container_1452791803262_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:43,513 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791803262_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E5E2B7825E4C4D62AA87194E88F19494, taskAttemptId=attempt_1452791803262_0001_1_01_000000_0, startTime=1452791803513, containerId=container_1452791803262_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791803262_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:43,513 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:43 Starting to run new task attempt: attempt_1452791803262_0001_1_01_000000_0
2016-01-14 17:16:43,514 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791803262_0001_1_01_000000_0
2016-01-14 17:16:43,514 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:43,513 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791803262_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:43,514 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:43,514 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:43,514 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791803262_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:43,514 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:43,514 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:43,514 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:43,515 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [3FB9BBF3D05540678AC3B1088B65152A/A90F0F23B2674A0390336D55E56DE6B4] every/(1/1) ...lyplatform/ALL_'field'_-1, VertexName: E5E2B7825E4C4D62AA87194E88F19494, VertexParallelism: 1, TaskAttemptID:attempt_1452791803262_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D0276CE57BE2453A92A2C2985821152E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=8829B94C076A4C9BA7B450B80B4D8BB4, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:43,515 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=716177408
2016-01-14 17:16:43,515 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791803262_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:43,515 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791803262_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:43,515 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791803262_0001_1_01_000000_0
2016-01-14 17:16:43,515 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:43,515 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791803262_0001_00_000001, Exiting
2016-01-14 17:16:43,516 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791803262_0001_00_000001
2016-01-14 17:16:43,516 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791803262_0001_00_000001
2016-01-14 17:16:43,516 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791803262_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:43,516 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:43,517 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791803262_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:43,517 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791803262_0001_00_000001 completed successfully
2016-01-14 17:16:43,517 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=D0276CE57BE2453A92A2C2985821152E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:43,517 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791803262_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:43,517 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:43,517 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: D0276CE57BE2453A92A2C2985821152E
2016-01-14 17:16:43,518 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:43,518 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=8829B94C076A4C9BA7B450B80B4D8BB4, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:43,518 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:43,518 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 8829B94C076A4C9BA7B450B80B4D8BB4
2016-01-14 17:16:43,518 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:43,518 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 8829B94C076A4C9BA7B450B80B4D8BB4
2016-01-14 17:16:43,520 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:43,524 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 644559680, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:43,524 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:43,525 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 644559680, type: INPUT, componentVertexName: D0276CE57BE2453A92A2C2985821152E
2016-01-14 17:16:43,525 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: D0276CE57BE2453A92A2C2985821152E
2016-01-14 17:16:43,527 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E5E2B7825E4C4D62AA87194E88F19494, ordinal: 1
2016-01-14 17:16:43,527 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E5E2B7825E4C4D62AA87194E88F19494, mem on start (mb), free: 509, total: 683, max: 683
2016-01-14 17:16:43,527 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:43,527 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:43,528 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:43,528 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:43,528 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 8829B94C076A4C9BA7B450B80B4D8BB4
2016-01-14 17:16:43,528 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:43,528 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:43,529 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:43,529 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:43,529 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:43,529 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:43,529 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 644559680, TotalRequestedScaled: 5.949781661538461E8, TotalJVMHeap: 716177408, TotalAvailable: 479838863, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:43,529 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 8829B94C076A4C9BA7B450B80B4D8BB4, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:43,529 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, D0276CE57BE2453A92A2C2985821152E, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=644559680, allocated=479838863
2016-01-14 17:16:43,529 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:43,530 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: D0276CE57BE2453A92A2C2985821152E being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:43,530 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:43,530 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:43,530 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: D0276CE57BE2453A92A2C2985821152E
2016-01-14 17:16:43,531 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:43,532 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:43,532 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: D0276CE57BE2453A92A2C2985821152E with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:43,532 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=644559680, postMergeMem=0, RuntimeTotalAvailable=479838863. Updated to: ShuffleMem=479838863, postMergeMem: 0
2016-01-14 17:16:43,532 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=479838863, maxSingleShuffleLimit=119959712, mergeThreshold=431854944, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:43,532 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:43,533 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: D0276CE57BE2453A92A2C2985821152E
2016-01-14 17:16:43,533 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:43,534 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791803262_0001_1_01_000000_0
2016-01-14 17:16:43,535 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791803262_0001_1_00_000000_0_10231, runDuration: 33219, ]
2016-01-14 17:16:43,536 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791803262_0001_1_00_000000_0_10231] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791803536, TimeTaken=0, Rate=0.00 MB/s
2016-01-14 17:16:43,536 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : D0276CE57BE2453A92A2C2985821152E
2016-01-14 17:16:43,536 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:43,537 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:43,537 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [D0276CE57BE2453A92A2C2985821152E] #39 in 1ms
2016-01-14 17:16:43,537 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 6E06FAEB221F4C61BE325799909811B6
2016-01-14 17:16:43,537 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [D0276CE57BE2453A92A2C2985821152E] #39, status:false, isInterrupted:false
2016-01-14 17:16:43,537 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:43,537 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_-1"], id: 8829B94C076A4C9BA7B450B80B4D8BB4
2016-01-14 17:16:43,538 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:43,538 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:43,538 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:43,538 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_-1"]
2016-01-14 17:16:43,538 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:43,538 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : D0276CE57BE2453A92A2C2985821152E
2016-01-14 17:16:43,538 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:43,538 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 6E06FAEB221F4C61BE325799909811B6, for 1 inputs
2016-01-14 17:16:43,539 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E5E2B7825E4C4D62AA87194E88F19494, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:43,540 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@168c28cc; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@4bca74a4
2016-01-14 17:16:43,544 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:43,544 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:43,545 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791803262_0001_1_01_000000_0
2016-01-14 17:16:43,545 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791803262_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:43,546 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791803262_0001_1_01_000000_0
2016-01-14 17:16:43,546 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E5E2B7825E4C4D62AA87194E88F19494, mem on close (mb), free: 505, total: 683, max: 683
2016-01-14 17:16:43,546 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: D0276CE57BE2453A92A2C2985821152E
2016-01-14 17:16:43,546 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:43,547 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:43,547 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:43,547 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:43,547 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:43 Completed running task attempt: attempt_1452791803262_0001_1_01_000000_0
2016-01-14 17:16:43,548 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791803262_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:43,548 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=716177408, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791803539, Process_Duration=7, Process_End_Time=1452791803546, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 17:16:43,548 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791803262_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E5E2B7825E4C4D62AA87194E88F19494, taskAttemptId=attempt_1452791803262_0001_1_01_000000_0, startTime=1452791803513, finishTime=1452791803548, timeTaken=35, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=716177408, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791803539, Process_Duration=7, Process_End_Time=1452791803546, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:43,548 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:43,549 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791803262_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:43,549 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:43,549 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:43,549 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791803262_0001_1_01_000000_0
2016-01-14 17:16:43,549 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:43,550 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791803262_0001_00_000002
2016-01-14 17:16:43,550 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791803262_0001_1][Event:TASK_FINISHED]: vertexName=E5E2B7825E4C4D62AA87194E88F19494, taskId=task_1452791803262_0001_1_01_000000, startTime=1452791803513, finishTime=1452791803549, timeTaken=36, status=SUCCEEDED, successfulAttemptID=attempt_1452791803262_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=716177408, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791803539, Process_Duration=7, Process_End_Time=1452791803546, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:43,550 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:43,550 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791803262_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:43,550 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791803262_0001_1_01 [E5E2B7825E4C4D62AA87194E88F19494] : 1
2016-01-14 17:16:43,550 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791803262_0001_1_01 [E5E2B7825E4C4D62AA87194E88F19494]
2016-01-14 17:16:43,551 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:43,551 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791803262_0001_1][Event:VERTEX_FINISHED]: vertexName=E5E2B7825E4C4D62AA87194E88F19494, vertexId=vertex_1452791803262_0001_1_01, initRequestedTime=1452791803428, initedTime=1452791803435, startRequestedTime=1452791803442, startedTime=1452791803442, finishTime=1452791803550, timeTaken=108, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=716177408, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791803539, Process_Duration=7, Process_End_Time=1452791803546, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452791803513, firstTasksToStart=[ task_1452791803262_0001_1_01_000000 ], lastTaskFinishTime=1452791803548, lastTasksToFinish=[ task_1452791803262_0001_1_01_000000 ], minTaskDuration=35, maxTaskDuration=35, avgTaskDuration=35.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791803262_0001_1_01_000000 ], longestDurationTasks=[ task_1452791803262_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:43,551 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791803262_0001_1_01 [E5E2B7825E4C4D62AA87194E88F19494] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:43,551 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791803262_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:43,551 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791803262_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791803262_0001_00_000002, stoppedTime=1452791803551, exitStatus=0
2016-01-14 17:16:43,551 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:43,551 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791803262_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:43,552 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791803262_0001_1_01 [E5E2B7825E4C4D62AA87194E88F19494] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:43,552 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791803262_0001_1
2016-01-14 17:16:43,552 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791803262_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452791803262_0001_1
2016-01-14 17:16:43,552 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452791803262_0001_1_00 [D0276CE57BE2453A92A2C2985821152E]
2016-01-14 17:16:43,552 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 8829B94C076A4C9BA7B450B80B4D8BB4 for vertex: vertex_1452791803262_0001_1_01 [E5E2B7825E4C4D62AA87194E88F19494]
2016-01-14 17:16:43,560 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791803262_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 17:16:43,560 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791803262_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:43,561 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791803262_0001_00_000002, Exiting
2016-01-14 17:16:43,561 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791803262_0001_00_000002
2016-01-14 17:16:43,561 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791803262_0001_00_000002
2016-01-14 17:16:43,561 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791803262_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:43,561 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791803262_0001_00_000002 completed successfully
2016-01-14 17:16:43,562 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791803262_0001_1, queueSize=0
2016-01-14 17:16:43,562 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:43,562 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791803262_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791803262_0001_1, startTime=1452791803424, finishTime=1452791803558, timeTaken=134, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=465, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1432354816, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905583607034, Process_Duration=10, Process_End_Time=2905583607044, Read_Duration=2, Tuples_Read=40, Tuples_Written=40, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 17:16:43,562 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791803262_0001_1 finished with state: SUCCEEDED
2016-01-14 17:16:43,562 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791803262_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:43,563 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791803262_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:16:43,563 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452791803262_0001_00_000002
2016-01-14 17:16:43 Completed Dag: dag_1452791803262_0001_1
2016-01-14 17:16:43,564 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791803262_0001_1, dagState=SUCCEEDED
2016-01-14 17:16:43,564 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791803262_0001_1_post
2016-01-14 17:16:43,565 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:43,565 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:43,569 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...lyplatform/ALL_'field'_-1, applicationId=application_1452791803262_0001
2016-01-14 17:16:43,571 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:43,571 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:43,571 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:43,571 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:43,571 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:43,572 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:43,572 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:43,572 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:43,572 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:43,572 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:43,573 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:43,573 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:43,573 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:43,580 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3FB9BBF3D05540678AC3B1088B65152A/A90F0F23B2674A0390336D55E56DE6B4/.tez/application_1452791803262_0001
2016-01-14 17:16:43,581 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.337
2016-01-14 17:16:43,641 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_'field'_-1
2016-01-14 17:16:43,661 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:43,661 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:43,689 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.006
2016-01-14 17:16:43,698 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-01-14 17:16:43,699 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:43,699 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:43,699 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:43,717 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:43,724 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_'field'
2016-01-14 17:16:43,754 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 17:16:43,754 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 17:16:43,769 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:43,769 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:43,770 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_'field'"]
2016-01-14 17:16:43,770 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:43,771 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:43,771 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:43,772 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...tform/ALL_'field'_'field'
2016-01-14 17:16:43,772 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:43,782 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:43,786 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:43,787 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:43,787 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:43,796 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/197634E53362493D93546D4D5CE7E9E0/B94618AF99D84A5797AFAFEE4E30C124/.tez/application_1452791803787_0001 doesn't exist and is created
2016-01-14 17:16:43,810 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:43,811 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:43,812 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/197634E53362493D93546D4D5CE7E9E0/B94618AF99D84A5797AFAFEE4E30C124/.tez/application_1452791803787_0001
2016-01-14 17:16:43,814 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791803787_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:43,829 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:43,830 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:43,831 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:43,831 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:43,831 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/197634E53362493D93546D4D5CE7E9E0/B94618AF99D84A5797AFAFEE4E30C124/.tez/application_1452791803787_0001/localmode-log-dir
2016-01-14 17:16:43,832 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/197634E53362493D93546D4D5CE7E9E0/B94618AF99D84A5797AFAFEE4E30C124/.tez/application_1452791803787_0001/localmode-log-dir/history.txt.appattempt_1452791803787_0001_000000, maxErrors=10
2016-01-14 17:16:43,832 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:43,832 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791803787_0001_000000, appSubmitTime=1452791803814, launchTime=1452791803814
2016-01-14 17:16:43,839 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:43,840 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:43,841 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:56329
2016-01-14 17:16:43,842 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:43,842 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:43,843 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:43,844 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791803787_0001_000000, startTime=1452791803844
2016-01-14 17:16:43,844 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:43,844 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:43,911 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:43,911 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:43,911 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...tform/ALL_'field'_'field', applicationId=application_1452791803787_0001, dagName=[197634E53362493D93546D4D5CE7E9E0/B94618AF99D84A5797AFAFEE4E30C124] every/(1/1) ...tform/ALL_'field'_'field'
2016-01-14 17:16:43,925 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [197634E53362493D93546D4D5CE7E9E0/B94618AF99D84A5797AFAFEE4E30C124] every/(1/1) ...tform/ALL_'field'_'field'
2016-01-14 17:16:43,926 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791803787_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/197634E53362493D93546D4D5CE7E9E0/B94618AF99D84A5797AFAFEE4E30C124/.tez/application_1452791803787_0001/localmode-log-dir/dag_1452791803787_0001_1.dot
2016-01-14 17:16:43,927 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/197634E53362493D93546D4D5CE7E9E0/B94618AF99D84A5797AFAFEE4E30C124/.tez/application_1452791803787_0001/localmode-log-dir/dag_1452791803787_0001_1-tez-dag.pb.txt
2016-01-14 17:16:43,932 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791803787_0001_1
2016-01-14 17:16:43,932 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:43,932 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [197634E53362493D93546D4D5CE7E9E0/B94618AF99D84A5797AFAFEE4E30C124] every/(1/1) ...tform/ALL_'field'_'field'
2016-01-14 17:16:43 Running Dag: dag_1452791803787_0001_1
2016-01-14 17:16:43,943 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791803787_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791803787_0001_1, submitTime=1452791803925
2016-01-14 17:16:43,943 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:43,943 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex B14D570B467940E0ADB7C9FA46EE76C6
2016-01-14 17:16:43,944 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:43,944 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791803787_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791803787_0001_1, initTime=1452791803943
2016-01-14 17:16:43,944 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:43,944 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791803787_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:43,944 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791803787_0001_1][Event:DAG_STARTED]: dagID=dag_1452791803787_0001_1, startTime=1452791803944
2016-01-14 17:16:43,945 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:43,945 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791803787_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:43,945 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: F1735678F53E4512964FD2B91CE1489D : {32EC7BA617BF4EBC82452B7AE076706A={InputName=32EC7BA617BF4EBC82452B7AE076706A}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:43,945 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 32EC7BA617BF4EBC82452B7AE076706A, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:43,945 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791803787_0001_1_00 [F1735678F53E4512964FD2B91CE1489D]
2016-01-14 17:16:43,945 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791803787_0001_1_00 [F1735678F53E4512964FD2B91CE1489D]
2016-01-14 17:16:43,945 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791803787_0001_1_00 [F1735678F53E4512964FD2B91CE1489D]
2016-01-14 17:16:43,946 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791803787_0001_1_00 [F1735678F53E4512964FD2B91CE1489D]. Starting root input initializers: 1
2016-01-14 17:16:43,945 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...tform/ALL_'field'_'field', applicationId=application_1452791803787_0001, dagName=[197634E53362493D93546D4D5CE7E9E0/B94618AF99D84A5797AFAFEE4E30C124] every/(1/1) ...tform/ALL_'field'_'field'
2016-01-14 17:16:43,946 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791803787_0001_1_00 [F1735678F53E4512964FD2B91CE1489D] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:43,946 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791803787_0001_1_01 [B14D570B467940E0ADB7C9FA46EE76C6]
2016-01-14 17:16:43,945 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:43,948 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 32EC7BA617BF4EBC82452B7AE076706A on vertex vertex_1452791803787_0001_1_00 [F1735678F53E4512964FD2B91CE1489D]
2016-01-14 17:16:43,949 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:43,949 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791803787_0001_1_01 [B14D570B467940E0ADB7C9FA46EE76C6]
2016-01-14 17:16:43,949 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791803787_0001_1_01 [B14D570B467940E0ADB7C9FA46EE76C6]
2016-01-14 17:16:43,949 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791803787_0001_1_01 [B14D570B467940E0ADB7C9FA46EE76C6]
2016-01-14 17:16:43,950 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=F0337B5B99284DEEA268D773496A4BC5, vertexId=vertex_1452791803787_0001_1_01 [B14D570B467940E0ADB7C9FA46EE76C6], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:43,950 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=F0337B5B99284DEEA268D773496A4BC5, vertexId=vertex_1452791803787_0001_1_01 [B14D570B467940E0ADB7C9FA46EE76C6]
2016-01-14 17:16:43,951 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:43,951 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 32EC7BA617BF4EBC82452B7AE076706A asking for 1 tasks. Headroom: 684 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:43,952 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=F0337B5B99284DEEA268D773496A4BC5, vertexName=B14D570B467940E0ADB7C9FA46EE76C6, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:43,952 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=F0337B5B99284DEEA268D773496A4BC5, vertexId=vertex_1452791803787_0001_1_01 [B14D570B467940E0ADB7C9FA46EE76C6]
2016-01-14 17:16:43,952 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:43,953 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:43,953 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:43,953 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791803787_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B14D570B467940E0ADB7C9FA46EE76C6, vertexId=vertex_1452791803787_0001_1_01, initRequestedTime=1452791803946, initedTime=1452791803953, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:43,953 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791803787_0001_1_01 [B14D570B467940E0ADB7C9FA46EE76C6] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:43,955 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:43,955 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 32EC7BA617BF4EBC82452B7AE076706A on vertex vertex_1452791803787_0001_1_00 [F1735678F53E4512964FD2B91CE1489D]
2016-01-14 17:16:43,955 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791803787_0001_1_00 [F1735678F53E4512964FD2B91CE1489D] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:43,956 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {32EC7BA617BF4EBC82452B7AE076706A=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:43,956 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791803787_0001_1_00 [F1735678F53E4512964FD2B91CE1489D] parallelism set to 1
2016-01-14 17:16:43,956 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791803787_0001_1_00 [F1735678F53E4512964FD2B91CE1489D] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:43,956 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791803787_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791803787_0001_1_00, eventCount=1
2016-01-14 17:16:43,956 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791803787_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F1735678F53E4512964FD2B91CE1489D, vertexId=vertex_1452791803787_0001_1_00, initRequestedTime=1452791803945, initedTime=1452791803956, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:43,957 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791803787_0001_1_00 [F1735678F53E4512964FD2B91CE1489D] with distanceFromRoot: 0
2016-01-14 17:16:43,956 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:43,957 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791803787_0001_1_00 [F1735678F53E4512964FD2B91CE1489D] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:43,957 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in F1735678F53E4512964FD2B91CE1489D
2016-01-14 17:16:43,957 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791803787_0001_1_00 [F1735678F53E4512964FD2B91CE1489D]
2016-01-14 17:16:43,958 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:43,958 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791803787_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791803787_0001_1_00, startRequestedTime=1452791803957, startedTime=1452791803957
2016-01-14 17:16:43,958 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791803787_0001_1_00 [F1735678F53E4512964FD2B91CE1489D] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:43,958 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:43,959 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791803787_0001_1][Event:TASK_STARTED]: vertexName=F1735678F53E4512964FD2B91CE1489D, taskId=task_1452791803787_0001_1_00_000000, scheduledTime=1452791803958, launchTime=1452791803958
2016-01-14 17:16:43,959 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791803787_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:43,959 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791803787_0001_1_00 for vertex: vertex_1452791803787_0001_1_01 [B14D570B467940E0ADB7C9FA46EE76C6] numStartedSources: 1 numSources: 1
2016-01-14 17:16:43,959 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791803787_0001_1_01 [B14D570B467940E0ADB7C9FA46EE76C6] with distanceFromRoot: 1
2016-01-14 17:16:43,959 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791803787_0001_1_00 [F1735678F53E4512964FD2B91CE1489D]
2016-01-14 17:16:43,960 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: F1735678F53E4512964FD2B91CE1489D in vertex: B14D570B467940E0ADB7C9FA46EE76C6
2016-01-14 17:16:43,960 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: B14D570B467940E0ADB7C9FA46EE76C6 with 1 source tasks and 1 pending tasks
2016-01-14 17:16:43,960 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:43,961 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791803787_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791803787_0001_1_01, startRequestedTime=1452791803959, startedTime=1452791803959
2016-01-14 17:16:43,961 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791803787_0001_1_01 [B14D570B467940E0ADB7C9FA46EE76C6] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:43,961 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [197634E53362493D93546D4D5CE7E9E0/B94618AF99D84A5797AFAFEE4E30C124] every/(1/1) ...tform/ALL_'field'_'field', VertexName: F1735678F53E4512964FD2B91CE1489D, VertexParallelism: 1, TaskAttemptID:attempt_1452791803787_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=32EC7BA617BF4EBC82452B7AE076706A, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B14D570B467940E0ADB7C9FA46EE76C6, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:43,961 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791803787_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:43,961 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:43,962 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:43,962 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791803787_0001, with dag id: dag_1452791803787_0001_1
2016-01-14 17:16:43,962 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:43,963 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:43,963 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791803787_0001_00_000001
2016-01-14 17:16:43,963 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791803787_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:43,964 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791803787_0001_00_000001
2016-01-14 17:16:43,964 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:43,965 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791803787_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:43,965 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:43,965 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791803787_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791803787_0001_00_000001, launchTime=1452791803965
2016-01-14 17:16:43,966 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791803787_0001_00_000001
2016-01-14 17:16:43,967 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791803787_0001_1_00_000000_0] to container: [container_1452791803787_0001_00_000001]
2016-01-14 17:16:43,967 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791803787_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:43,967 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791803787_0001_1_00_000000_0] started. Is using containerId: [container_1452791803787_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:43,967 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791803787_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F1735678F53E4512964FD2B91CE1489D, taskAttemptId=attempt_1452791803787_0001_1_00_000000_0, startTime=1452791803967, containerId=container_1452791803787_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791803787_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:43,967 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791803787_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:43,967 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791803787_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:43,967 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:43,968 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791803787_0001_00_000001 given task: attempt_1452791803787_0001_1_00_000000_0
2016-01-14 17:16:43,968 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791803787_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791803787_0001_1_00_000000_0
2016-01-14 17:16:43 Starting to run new task attempt: attempt_1452791803787_0001_1_00_000000_0
2016-01-14 17:16:43,968 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791803787_0001_1_00_000000_0
2016-01-14 17:16:43,968 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:43,968 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:43,968 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:43,969 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:43,969 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:43,969 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:43,969 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [197634E53362493D93546D4D5CE7E9E0/B94618AF99D84A5797AFAFEE4E30C124] every/(1/1) ...tform/ALL_'field'_'field', VertexName: F1735678F53E4512964FD2B91CE1489D, VertexParallelism: 1, TaskAttemptID:attempt_1452791803787_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=32EC7BA617BF4EBC82452B7AE076706A, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B14D570B467940E0ADB7C9FA46EE76C6, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:43,969 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=717225984
2016-01-14 17:16:43,970 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791803787_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:43,970 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791803787_0001_1_00_000000_0
2016-01-14 17:16:43,970 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:43,970 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:43,971 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=32EC7BA617BF4EBC82452B7AE076706A, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:43,971 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=B14D570B467940E0ADB7C9FA46EE76C6, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:43,971 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:43,971 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: B14D570B467940E0ADB7C9FA46EE76C6
2016-01-14 17:16:43,971 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:43,971 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:43,972 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 32EC7BA617BF4EBC82452B7AE076706A
2016-01-14 17:16:43,973 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 32EC7BA617BF4EBC82452B7AE076706A
2016-01-14 17:16:43,974 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:43,974 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: B14D570B467940E0ADB7C9FA46EE76C6
2016-01-14 17:16:43,974 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: B14D570B467940E0ADB7C9FA46EE76C6
2016-01-14 17:16:43,975 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:43,975 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 32EC7BA617BF4EBC82452B7AE076706A
2016-01-14 17:16:43,975 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 32EC7BA617BF4EBC82452B7AE076706A
2016-01-14 17:16:43,975 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:43,978 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F1735678F53E4512964FD2B91CE1489D, ordinal: 0
2016-01-14 17:16:43,978 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F1735678F53E4512964FD2B91CE1489D, mem on start (mb), free: 609, total: 684, max: 684
2016-01-14 17:16:43,978 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:43,978 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:43,978 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:43,978 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:43,978 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:43,978 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:43,979 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:43,979 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:43,979 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:43,979 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 717225984, TotalAvailable: 480541409, TotalRequested/TotalJVMHeap:0.15
2016-01-14 17:16:43,979 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 32EC7BA617BF4EBC82452B7AE076706A, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:43,979 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, B14D570B467940E0ADB7C9FA46EE76C6, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:43,979 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:43,979 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 32EC7BA617BF4EBC82452B7AE076706A
2016-01-14 17:16:43,979 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:43,979 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 32EC7BA617BF4EBC82452B7AE076706A
2016-01-14 17:16:43,979 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 32EC7BA617BF4EBC82452B7AE076706A being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:43,980 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:43,980 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:43,980 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:43,980 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791803787_0001_1_00_000000_0
2016-01-14 17:16:43,983 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:43,983 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:43,984 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 32EC7BA617BF4EBC82452B7AE076706A
2016-01-14 17:16:43,984 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 9D13DF192CAE418F821C7F1DB9FD4A97
2016-01-14 17:16:43,984 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 9D13DF192CAE418F821C7F1DB9FD4A97
2016-01-14 17:16:43,984 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@1fa0a562
2016-01-14 17:16:43,984 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@6bc9d526; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@fc8e913; comparator=cascading.tuple.hadoop.util.TupleComparator@4f3de67d; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:43,985 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:43,985 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:43,985 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:43,986 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@1fa0a562
2016-01-14 17:16:43,986 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:43,987 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:43,998 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:43,998 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:43,998 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:43,998 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:43,998 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:43,999 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:43,999 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:43,999 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F1735678F53E4512964FD2B91CE1489D, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:44,002 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791803787_0001_1_00_000000_0
2016-01-14 17:16:44,002 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F1735678F53E4512964FD2B91CE1489D, mem on close (mb), free: 507, total: 684, max: 684
2016-01-14 17:16:44,002 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:44,003 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:44,003 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:44,003 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:44,005 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:44 Completed running task attempt: attempt_1452791803787_0001_1_00_000000_0
2016-01-14 17:16:44,008 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791803787_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:44,008 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791803787_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791803787_0001_1_00, eventCount=1
2016-01-14 17:16:44,008 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791803787_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F1735678F53E4512964FD2B91CE1489D, taskAttemptId=attempt_1452791803787_0001_1_00_000000_0, startTime=1452791803967, finishTime=1452791804008, timeTaken=41, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=717225984, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791803999, Process_Duration=3, Process_End_Time=1452791804002, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:44,008 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791803787_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:44,009 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:44,009 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791803787_0001_1_00_000000_0
2016-01-14 17:16:44,009 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:44,009 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791803787_0001_1][Event:TASK_FINISHED]: vertexName=F1735678F53E4512964FD2B91CE1489D, taskId=task_1452791803787_0001_1_00_000000, startTime=1452791803967, finishTime=1452791804009, timeTaken=42, status=SUCCEEDED, successfulAttemptID=attempt_1452791803787_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=717225984, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791803999, Process_Duration=3, Process_End_Time=1452791804002, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:44,009 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:44,010 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791803787_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:44,010 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791803787_0001_1_00 [F1735678F53E4512964FD2B91CE1489D] : 1
2016-01-14 17:16:44,010 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791803787_0001_1_00 [F1735678F53E4512964FD2B91CE1489D]
2016-01-14 17:16:44,010 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=717225984, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791803999, Process_Duration=3, Process_End_Time=1452791804002, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:44,010 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:44,010 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:44,010 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:44,010 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791803787_0001_1][Event:VERTEX_FINISHED]: vertexName=F1735678F53E4512964FD2B91CE1489D, vertexId=vertex_1452791803787_0001_1_00, initRequestedTime=1452791803945, initedTime=1452791803956, startRequestedTime=1452791803957, startedTime=1452791803957, finishTime=1452791804010, timeTaken=53, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=717225984, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791803999, Process_Duration=3, Process_End_Time=1452791804002, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791803967, firstTasksToStart=[ task_1452791803787_0001_1_00_000000 ], lastTaskFinishTime=1452791804008, lastTasksToFinish=[ task_1452791803787_0001_1_00_000000 ], minTaskDuration=41, maxTaskDuration=41, avgTaskDuration=41.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791803787_0001_1_00_000000 ], longestDurationTasks=[ task_1452791803787_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:44,011 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791803787_0001_1_00 [F1735678F53E4512964FD2B91CE1489D] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:44,011 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791803787_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:44,011 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791803787_0001_00_000001
2016-01-14 17:16:44,011 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791803787_0001_1_01 [B14D570B467940E0ADB7C9FA46EE76C6] attempt: attempt_1452791803787_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:44,011 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: B14D570B467940E0ADB7C9FA46EE76C6
2016-01-14 17:16:44,011 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791803787_0001_1_01 [B14D570B467940E0ADB7C9FA46EE76C6]
2016-01-14 17:16:44,011 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791803787_0001_1_00 [F1735678F53E4512964FD2B91CE1489D] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:44,011 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791803787_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791803787_0001_00_000001, stoppedTime=1452791804011, exitStatus=0
2016-01-14 17:16:44,011 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:44,011 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791803787_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:44,012 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791803787_0001_1][Event:TASK_STARTED]: vertexName=B14D570B467940E0ADB7C9FA46EE76C6, taskId=task_1452791803787_0001_1_01_000000, scheduledTime=1452791804012, launchTime=1452791804012
2016-01-14 17:16:44,012 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:44,012 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791803787_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:44,012 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791803787_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:44,013 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791803787_0001_1_01 [B14D570B467940E0ADB7C9FA46EE76C6]
2016-01-14 17:16:44,013 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791803787_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:44,013 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [197634E53362493D93546D4D5CE7E9E0/B94618AF99D84A5797AFAFEE4E30C124] every/(1/1) ...tform/ALL_'field'_'field', VertexName: B14D570B467940E0ADB7C9FA46EE76C6, VertexParallelism: 1, TaskAttemptID:attempt_1452791803787_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F1735678F53E4512964FD2B91CE1489D, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=F0337B5B99284DEEA268D773496A4BC5, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:44,013 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791803787_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:44,013 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:44,014 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791803787_0001_00_000002
2016-01-14 17:16:44,014 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791803787_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:44,016 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791803787_0001_00_000002
2016-01-14 17:16:44,016 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:44,016 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791803787_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:44,016 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:44,017 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791803787_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791803787_0001_00_000002, launchTime=1452791804016
2016-01-14 17:16:44,017 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791803787_0001_00_000002
2016-01-14 17:16:44,017 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791803787_0001_1_01_000000_0] to container: [container_1452791803787_0001_00_000002]
2016-01-14 17:16:44,018 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791803787_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:44,018 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791803787_0001_00_000002 given task: attempt_1452791803787_0001_1_01_000000_0
2016-01-14 17:16:44,018 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791803787_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791803787_0001_1_01_000000_0
2016-01-14 17:16:44,018 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791803787_0001_1_01_000000_0] started. Is using containerId: [container_1452791803787_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:44,018 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791803787_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B14D570B467940E0ADB7C9FA46EE76C6, taskAttemptId=attempt_1452791803787_0001_1_01_000000_0, startTime=1452791804018, containerId=container_1452791803787_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791803787_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:44,018 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:44,018 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791803787_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:44 Starting to run new task attempt: attempt_1452791803787_0001_1_01_000000_0
2016-01-14 17:16:44,018 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791803787_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:44,018 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791803787_0001_1_01_000000_0
2016-01-14 17:16:44,019 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:44,019 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:44,019 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:44,019 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:44,019 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:44,019 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:44,019 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [197634E53362493D93546D4D5CE7E9E0/B94618AF99D84A5797AFAFEE4E30C124] every/(1/1) ...tform/ALL_'field'_'field', VertexName: B14D570B467940E0ADB7C9FA46EE76C6, VertexParallelism: 1, TaskAttemptID:attempt_1452791803787_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F1735678F53E4512964FD2B91CE1489D, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=F0337B5B99284DEEA268D773496A4BC5, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:44,020 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=717225984
2016-01-14 17:16:44,020 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791803787_0001_1_01_000000_0
2016-01-14 17:16:44,020 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:44,020 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:44,021 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791803787_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:44,021 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F1735678F53E4512964FD2B91CE1489D, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:44,021 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791803787_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:44,021 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:44,021 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791803787_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:44,021 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F1735678F53E4512964FD2B91CE1489D
2016-01-14 17:16:44,021 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791803787_0001_00_000001, Exiting
2016-01-14 17:16:44,022 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791803787_0001_00_000001
2016-01-14 17:16:44,022 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791803787_0001_00_000001
2016-01-14 17:16:44,023 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 645503360, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:44,023 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:44,023 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 645503360, type: INPUT, componentVertexName: F1735678F53E4512964FD2B91CE1489D
2016-01-14 17:16:44,023 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F1735678F53E4512964FD2B91CE1489D
2016-01-14 17:16:44,023 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:44,024 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791803787_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:44,024 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791803787_0001_00_000001 completed successfully
2016-01-14 17:16:44,025 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791803787_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:44,025 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:44,028 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B14D570B467940E0ADB7C9FA46EE76C6, ordinal: 1
2016-01-14 17:16:44,028 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B14D570B467940E0ADB7C9FA46EE76C6, mem on start (mb), free: 502, total: 684, max: 684
2016-01-14 17:16:44,028 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:44,028 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:44,028 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:44,028 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F0337B5B99284DEEA268D773496A4BC5, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:44,029 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:44,029 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F0337B5B99284DEEA268D773496A4BC5
2016-01-14 17:16:44,029 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:44,029 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: F0337B5B99284DEEA268D773496A4BC5
2016-01-14 17:16:44,036 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:44,036 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F0337B5B99284DEEA268D773496A4BC5
2016-01-14 17:16:44,036 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:44,036 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:44,036 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:44,036 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:44,036 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:44,036 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:44,037 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 645503360, TotalRequestedScaled: 5.958492553846154E8, TotalJVMHeap: 717225984, TotalAvailable: 480541409, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:44,037 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F1735678F53E4512964FD2B91CE1489D, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=645503360, allocated=480541409
2016-01-14 17:16:44,037 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F0337B5B99284DEEA268D773496A4BC5, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:44,037 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:44,037 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F1735678F53E4512964FD2B91CE1489D being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:44,037 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:44,037 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F1735678F53E4512964FD2B91CE1489D
2016-01-14 17:16:44,037 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:44,038 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:44,039 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:44,039 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: F1735678F53E4512964FD2B91CE1489D with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:44,039 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=645503360, postMergeMem=0, RuntimeTotalAvailable=480541409. Updated to: ShuffleMem=480541409, postMergeMem: 0
2016-01-14 17:16:44,039 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=480541409, maxSingleShuffleLimit=120135352, mergeThreshold=432487264, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:44,039 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:44,040 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F1735678F53E4512964FD2B91CE1489D
2016-01-14 17:16:44,040 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:44,040 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791803787_0001_1_01_000000_0
2016-01-14 17:16:44,042 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791803787_0001_1_00_000000_0_10237, runDuration: 35042, ]
2016-01-14 17:16:44,043 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:44,043 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 9D13DF192CAE418F821C7F1DB9FD4A97
2016-01-14 17:16:44,044 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791803787_0001_1_00_000000_0_10237] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791804043, TimeTaken=0, Rate=0.00 MB/s
2016-01-14 17:16:44,044 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : F1735678F53E4512964FD2B91CE1489D
2016-01-14 17:16:44,044 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_'field'"], id: F0337B5B99284DEEA268D773496A4BC5
2016-01-14 17:16:44,044 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_'field'"]
2016-01-14 17:16:44,044 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:44,044 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 9D13DF192CAE418F821C7F1DB9FD4A97, for 1 inputs
2016-01-14 17:16:44,046 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [F1735678F53E4512964FD2B91CE1489D] #40 in 3ms
2016-01-14 17:16:44,046 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [F1735678F53E4512964FD2B91CE1489D] #40, status:false, isInterrupted:false
2016-01-14 17:16:44,046 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:44,047 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:44,047 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:44,047 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:44,047 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:44,047 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : F1735678F53E4512964FD2B91CE1489D
2016-01-14 17:16:44,047 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:44,047 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B14D570B467940E0ADB7C9FA46EE76C6, all 1 inputs ready in: 00:00:00.003
2016-01-14 17:16:44,048 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@3a615933; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@4d274136
2016-01-14 17:16:44,052 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:44,052 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:44,053 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791803787_0001_1_01_000000_0
2016-01-14 17:16:44,053 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791803787_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:44,053 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791803787_0001_1_01_000000_0
2016-01-14 17:16:44,053 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B14D570B467940E0ADB7C9FA46EE76C6, mem on close (mb), free: 498, total: 684, max: 684
2016-01-14 17:16:44,053 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: F1735678F53E4512964FD2B91CE1489D
2016-01-14 17:16:44,054 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:44,054 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:44,054 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:44,054 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:44,054 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:44 Completed running task attempt: attempt_1452791803787_0001_1_01_000000_0
2016-01-14 17:16:44,055 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791803787_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:44,055 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:44,055 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791803787_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B14D570B467940E0ADB7C9FA46EE76C6, taskAttemptId=attempt_1452791803787_0001_1_01_000000_0, startTime=1452791804018, finishTime=1452791804055, timeTaken=37, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=717225984, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791804047, Process_Duration=6, Process_End_Time=1452791804053, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:44,055 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791803787_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:44,055 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791803787_0001_1_01_000000_0
2016-01-14 17:16:44,055 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:44,056 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791803787_0001_1][Event:TASK_FINISHED]: vertexName=B14D570B467940E0ADB7C9FA46EE76C6, taskId=task_1452791803787_0001_1_01_000000, startTime=1452791804018, finishTime=1452791804055, timeTaken=37, status=SUCCEEDED, successfulAttemptID=attempt_1452791803787_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=717225984, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791804047, Process_Duration=6, Process_End_Time=1452791804053, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:44,056 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791803787_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:44,056 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791803787_0001_1_01 [B14D570B467940E0ADB7C9FA46EE76C6] : 1
2016-01-14 17:16:44,056 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791803787_0001_1_01 [B14D570B467940E0ADB7C9FA46EE76C6]
2016-01-14 17:16:44,056 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:44,056 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:44,056 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791803787_0001_1][Event:VERTEX_FINISHED]: vertexName=B14D570B467940E0ADB7C9FA46EE76C6, vertexId=vertex_1452791803787_0001_1_01, initRequestedTime=1452791803946, initedTime=1452791803953, startRequestedTime=1452791803959, startedTime=1452791803959, finishTime=1452791804056, timeTaken=97, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=717225984, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791804047, Process_Duration=6, Process_End_Time=1452791804053, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452791804018, firstTasksToStart=[ task_1452791803787_0001_1_01_000000 ], lastTaskFinishTime=1452791804055, lastTasksToFinish=[ task_1452791803787_0001_1_01_000000 ], minTaskDuration=37, maxTaskDuration=37, avgTaskDuration=37.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791803787_0001_1_01_000000 ], longestDurationTasks=[ task_1452791803787_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:44,056 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791803787_0001_1_01 [B14D570B467940E0ADB7C9FA46EE76C6] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:44,057 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=717225984, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791804047, Process_Duration=6, Process_End_Time=1452791804053, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 17:16:44,057 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:44,057 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:44,057 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791803787_0001_00_000002
2016-01-14 17:16:44,058 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791803787_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:44,058 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791803787_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791803787_0001_00_000002, stoppedTime=1452791804058, exitStatus=0
2016-01-14 17:16:44,058 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:44,058 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791803787_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:44,058 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791803787_0001_1_01 [B14D570B467940E0ADB7C9FA46EE76C6] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:44,058 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791803787_0001_1
2016-01-14 17:16:44,058 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791803787_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452791803787_0001_1
2016-01-14 17:16:44,058 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: F0337B5B99284DEEA268D773496A4BC5 for vertex: vertex_1452791803787_0001_1_01 [B14D570B467940E0ADB7C9FA46EE76C6]
2016-01-14 17:16:44,061 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(221)) - No more tasks for container with id : container_1452791803787_0001_00_000002. Asking it to die
2016-01-14 17:16:44,062 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791803787_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:44,062 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791803787_0001_00_000002, Exiting
2016-01-14 17:16:44,062 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791803787_0001_00_000002
2016-01-14 17:16:44,062 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791803787_0001_00_000002
2016-01-14 17:16:44,062 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791803787_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:44,062 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791803787_0001_00_000002 completed successfully
2016-01-14 17:16:44,067 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452791803787_0001_1_00 [F1735678F53E4512964FD2B91CE1489D]
2016-01-14 17:16:44,070 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791803787_0001_1, queueSize=0
2016-01-14 17:16:44,070 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:44,071 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791803787_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791803787_0001_1, startTime=1452791803944, finishTime=1452791804067, timeTaken=123, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=465, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1434451968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905583608046, Process_Duration=9, Process_End_Time=2905583608055, Read_Duration=3, Tuples_Read=40, Tuples_Written=40, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 17:16:44,071 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791803787_0001_1 finished with state: SUCCEEDED
2016-01-14 17:16:44,071 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791803787_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:44,072 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791803787_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:16:44,072 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452791803787_0001_00_000002
2016-01-14 17:16:44 Completed Dag: dag_1452791803787_0001_1
2016-01-14 17:16:44,076 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791803787_0001_1, dagState=SUCCEEDED
2016-01-14 17:16:44,077 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791803787_0001_1_post
2016-01-14 17:16:44,077 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:44,077 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:44,077 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...tform/ALL_'field'_'field', applicationId=application_1452791803787_0001
2016-01-14 17:16:44,079 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:44,079 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:44,079 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:44,079 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:44,079 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:44,079 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:44,080 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:44,080 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:44,080 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:44,080 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:44,080 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:44,080 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:44,080 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:44,085 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/197634E53362493D93546D4D5CE7E9E0/B94618AF99D84A5797AFAFEE4E30C124/.tez/application_1452791803787_0001
2016-01-14 17:16:44,085 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.316
2016-01-14 17:16:44,145 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_'field'_'field'
2016-01-14 17:16:44,166 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:44,166 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:44,194 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.006
2016-01-14 17:16:44,201 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-01-14 17:16:44,201 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:44,202 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:44,202 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:44,215 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:44,221 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_RESULTS
2016-01-14 17:16:44,257 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 17:16:44,257 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 17:16:44,261 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:44,261 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:44,261 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_RESULTS"]
2016-01-14 17:16:44,262 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:44,262 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:44,262 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:44,263 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...tform/ALL_'field'_RESULTS
2016-01-14 17:16:44,264 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:44,283 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:44,288 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:44,288 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:44,288 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:44,297 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/150CC68B228D48C7A180A99C61A5ABEC/A97404B827D048BA817E079B533E6C3F/.tez/application_1452791804288_0001 doesn't exist and is created
2016-01-14 17:16:44,313 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:44,313 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:44,314 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/150CC68B228D48C7A180A99C61A5ABEC/A97404B827D048BA817E079B533E6C3F/.tez/application_1452791804288_0001
2016-01-14 17:16:44,317 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791804288_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:44,327 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:44,328 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:44,330 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:44,330 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:44,330 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/150CC68B228D48C7A180A99C61A5ABEC/A97404B827D048BA817E079B533E6C3F/.tez/application_1452791804288_0001/localmode-log-dir
2016-01-14 17:16:44,330 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/150CC68B228D48C7A180A99C61A5ABEC/A97404B827D048BA817E079B533E6C3F/.tez/application_1452791804288_0001/localmode-log-dir/history.txt.appattempt_1452791804288_0001_000000, maxErrors=10
2016-01-14 17:16:44,330 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:44,330 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791804288_0001_000000, appSubmitTime=1452791804316, launchTime=1452791804316
2016-01-14 17:16:44,336 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:44,336 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:44,336 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:56378
2016-01-14 17:16:44,337 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:44,339 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:44,340 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791804288_0001_000000, startTime=1452791804340
2016-01-14 17:16:44,341 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:44,344 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:44,344 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:44,414 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:44,414 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:44,414 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...tform/ALL_'field'_RESULTS, applicationId=application_1452791804288_0001, dagName=[150CC68B228D48C7A180A99C61A5ABEC/A97404B827D048BA817E079B533E6C3F] every/(1/1) ...tform/ALL_'field'_RESULTS
2016-01-14 17:16:44,427 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [150CC68B228D48C7A180A99C61A5ABEC/A97404B827D048BA817E079B533E6C3F] every/(1/1) ...tform/ALL_'field'_RESULTS
2016-01-14 17:16:44,429 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791804288_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/150CC68B228D48C7A180A99C61A5ABEC/A97404B827D048BA817E079B533E6C3F/.tez/application_1452791804288_0001/localmode-log-dir/dag_1452791804288_0001_1.dot
2016-01-14 17:16:44,429 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/150CC68B228D48C7A180A99C61A5ABEC/A97404B827D048BA817E079B533E6C3F/.tez/application_1452791804288_0001/localmode-log-dir/dag_1452791804288_0001_1-tez-dag.pb.txt
2016-01-14 17:16:44,435 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791804288_0001_1
2016-01-14 17:16:44,435 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:44,435 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [150CC68B228D48C7A180A99C61A5ABEC/A97404B827D048BA817E079B533E6C3F] every/(1/1) ...tform/ALL_'field'_RESULTS
2016-01-14 17:16:44 Running Dag: dag_1452791804288_0001_1
2016-01-14 17:16:44,447 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791804288_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791804288_0001_1, submitTime=1452791804428
2016-01-14 17:16:44,447 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:44,447 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex C2449BC657DB4D4BBE29CCAD5BCD9703
2016-01-14 17:16:44,447 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:44,448 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791804288_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791804288_0001_1, initTime=1452791804447
2016-01-14 17:16:44,448 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791804288_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:44,448 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:44,448 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791804288_0001_1][Event:DAG_STARTED]: dagID=dag_1452791804288_0001_1, startTime=1452791804448
2016-01-14 17:16:44,448 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:44,448 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:44,448 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791804288_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:44,448 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: B29662284148463C9012B2AB6C5AFC8F : {323E88D7FAF1428EBF0A65D58CD29981={InputName=323E88D7FAF1428EBF0A65D58CD29981}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:44,448 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 323E88D7FAF1428EBF0A65D58CD29981, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:44,448 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791804288_0001_1_00 [B29662284148463C9012B2AB6C5AFC8F]
2016-01-14 17:16:44,448 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791804288_0001_1_00 [B29662284148463C9012B2AB6C5AFC8F]
2016-01-14 17:16:44,448 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791804288_0001_1_00 [B29662284148463C9012B2AB6C5AFC8F]
2016-01-14 17:16:44,449 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791804288_0001_1_00 [B29662284148463C9012B2AB6C5AFC8F]. Starting root input initializers: 1
2016-01-14 17:16:44,449 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791804288_0001_1_00 [B29662284148463C9012B2AB6C5AFC8F] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:44,449 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791804288_0001_1_01 [C2449BC657DB4D4BBE29CCAD5BCD9703]
2016-01-14 17:16:44,449 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...tform/ALL_'field'_RESULTS, applicationId=application_1452791804288_0001, dagName=[150CC68B228D48C7A180A99C61A5ABEC/A97404B827D048BA817E079B533E6C3F] every/(1/1) ...tform/ALL_'field'_RESULTS
2016-01-14 17:16:44,450 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 323E88D7FAF1428EBF0A65D58CD29981 on vertex vertex_1452791804288_0001_1_00 [B29662284148463C9012B2AB6C5AFC8F]
2016-01-14 17:16:44,452 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:44,452 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791804288_0001_1_01 [C2449BC657DB4D4BBE29CCAD5BCD9703]
2016-01-14 17:16:44,452 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791804288_0001_1_01 [C2449BC657DB4D4BBE29CCAD5BCD9703]
2016-01-14 17:16:44,452 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:44,452 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791804288_0001_1_01 [C2449BC657DB4D4BBE29CCAD5BCD9703]
2016-01-14 17:16:44,452 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=FEB5F4D4E2AD4E73B209982A332B34B2, vertexId=vertex_1452791804288_0001_1_01 [C2449BC657DB4D4BBE29CCAD5BCD9703], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:44,452 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 323E88D7FAF1428EBF0A65D58CD29981 asking for 1 tasks. Headroom: 684 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:44,452 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=FEB5F4D4E2AD4E73B209982A332B34B2, vertexId=vertex_1452791804288_0001_1_01 [C2449BC657DB4D4BBE29CCAD5BCD9703]
2016-01-14 17:16:44,453 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:44,453 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:44,454 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=FEB5F4D4E2AD4E73B209982A332B34B2, vertexName=C2449BC657DB4D4BBE29CCAD5BCD9703, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:44,454 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=FEB5F4D4E2AD4E73B209982A332B34B2, vertexId=vertex_1452791804288_0001_1_01 [C2449BC657DB4D4BBE29CCAD5BCD9703]
2016-01-14 17:16:44,454 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:44,455 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 323E88D7FAF1428EBF0A65D58CD29981 on vertex vertex_1452791804288_0001_1_00 [B29662284148463C9012B2AB6C5AFC8F]
2016-01-14 17:16:44,455 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:44,455 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791804288_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C2449BC657DB4D4BBE29CCAD5BCD9703, vertexId=vertex_1452791804288_0001_1_01, initRequestedTime=1452791804449, initedTime=1452791804455, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:44,455 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791804288_0001_1_01 [C2449BC657DB4D4BBE29CCAD5BCD9703] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:44,456 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791804288_0001_1_00 [B29662284148463C9012B2AB6C5AFC8F] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:44,456 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {323E88D7FAF1428EBF0A65D58CD29981=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:44,456 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791804288_0001_1_00 [B29662284148463C9012B2AB6C5AFC8F] parallelism set to 1
2016-01-14 17:16:44,456 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791804288_0001_1_00 [B29662284148463C9012B2AB6C5AFC8F] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:44,456 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791804288_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791804288_0001_1_00, eventCount=1
2016-01-14 17:16:44,457 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791804288_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B29662284148463C9012B2AB6C5AFC8F, vertexId=vertex_1452791804288_0001_1_00, initRequestedTime=1452791804448, initedTime=1452791804456, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:44,457 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791804288_0001_1_00 [B29662284148463C9012B2AB6C5AFC8F] with distanceFromRoot: 0
2016-01-14 17:16:44,457 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:44,457 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791804288_0001_1_00 [B29662284148463C9012B2AB6C5AFC8F] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:44,458 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in B29662284148463C9012B2AB6C5AFC8F
2016-01-14 17:16:44,458 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791804288_0001_1_00 [B29662284148463C9012B2AB6C5AFC8F]
2016-01-14 17:16:44,459 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:44,459 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791804288_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791804288_0001_1_00, startRequestedTime=1452791804458, startedTime=1452791804458
2016-01-14 17:16:44,459 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791804288_0001_1_00 [B29662284148463C9012B2AB6C5AFC8F] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:44,459 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:44,460 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791804288_0001_1][Event:TASK_STARTED]: vertexName=B29662284148463C9012B2AB6C5AFC8F, taskId=task_1452791804288_0001_1_00_000000, scheduledTime=1452791804459, launchTime=1452791804459
2016-01-14 17:16:44,460 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791804288_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:44,460 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791804288_0001_1_00 for vertex: vertex_1452791804288_0001_1_01 [C2449BC657DB4D4BBE29CCAD5BCD9703] numStartedSources: 1 numSources: 1
2016-01-14 17:16:44,460 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791804288_0001_1_01 [C2449BC657DB4D4BBE29CCAD5BCD9703] with distanceFromRoot: 1
2016-01-14 17:16:44,460 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791804288_0001_1_00 [B29662284148463C9012B2AB6C5AFC8F]
2016-01-14 17:16:44,460 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: B29662284148463C9012B2AB6C5AFC8F in vertex: C2449BC657DB4D4BBE29CCAD5BCD9703
2016-01-14 17:16:44,461 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: C2449BC657DB4D4BBE29CCAD5BCD9703 with 1 source tasks and 1 pending tasks
2016-01-14 17:16:44,461 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:44,461 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791804288_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791804288_0001_1_01, startRequestedTime=1452791804460, startedTime=1452791804460
2016-01-14 17:16:44,461 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791804288_0001_1_01 [C2449BC657DB4D4BBE29CCAD5BCD9703] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:44,461 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [150CC68B228D48C7A180A99C61A5ABEC/A97404B827D048BA817E079B533E6C3F] every/(1/1) ...tform/ALL_'field'_RESULTS, VertexName: B29662284148463C9012B2AB6C5AFC8F, VertexParallelism: 1, TaskAttemptID:attempt_1452791804288_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=323E88D7FAF1428EBF0A65D58CD29981, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C2449BC657DB4D4BBE29CCAD5BCD9703, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:44,462 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791804288_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:44,462 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:44,462 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:44,463 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:44,463 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:44,463 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791804288_0001_00_000001
2016-01-14 17:16:44,463 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791804288_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:44,464 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791804288_0001_00_000001
2016-01-14 17:16:44,465 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:44,465 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791804288_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:44,465 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:44,466 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791804288_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791804288_0001_00_000001, launchTime=1452791804465
2016-01-14 17:16:44,466 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791804288_0001_00_000001
2016-01-14 17:16:44,467 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791804288_0001_1_00_000000_0] to container: [container_1452791804288_0001_00_000001]
2016-01-14 17:16:44,467 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791804288_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:44,467 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791804288_0001_1_00_000000_0] started. Is using containerId: [container_1452791804288_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:44,467 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791804288_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B29662284148463C9012B2AB6C5AFC8F, taskAttemptId=attempt_1452791804288_0001_1_00_000000_0, startTime=1452791804467, containerId=container_1452791804288_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791804288_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:44,468 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791804288_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:44,468 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:44,468 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791804288_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:44,468 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791804288_0001_00_000001 given task: attempt_1452791804288_0001_1_00_000000_0
2016-01-14 17:16:44,468 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791804288_0001, with dag id: dag_1452791804288_0001_1
2016-01-14 17:16:44,468 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791804288_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791804288_0001_1_00_000000_0
2016-01-14 17:16:44 Starting to run new task attempt: attempt_1452791804288_0001_1_00_000000_0
2016-01-14 17:16:44,469 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791804288_0001_1_00_000000_0
2016-01-14 17:16:44,469 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:44,469 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:44,469 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:44,469 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:44,469 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:44,469 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:44,469 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [150CC68B228D48C7A180A99C61A5ABEC/A97404B827D048BA817E079B533E6C3F] every/(1/1) ...tform/ALL_'field'_RESULTS, VertexName: B29662284148463C9012B2AB6C5AFC8F, VertexParallelism: 1, TaskAttemptID:attempt_1452791804288_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=323E88D7FAF1428EBF0A65D58CD29981, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C2449BC657DB4D4BBE29CCAD5BCD9703, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:44,470 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=717225984
2016-01-14 17:16:44,470 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791804288_0001_1_00_000000_0
2016-01-14 17:16:44,470 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:44,470 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791804288_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:44,470 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:44,471 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:44,471 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=C2449BC657DB4D4BBE29CCAD5BCD9703, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:44,472 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:44,472 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: C2449BC657DB4D4BBE29CCAD5BCD9703
2016-01-14 17:16:44,474 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:44,474 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:44,474 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: C2449BC657DB4D4BBE29CCAD5BCD9703
2016-01-14 17:16:44,474 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: C2449BC657DB4D4BBE29CCAD5BCD9703
2016-01-14 17:16:44,475 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=323E88D7FAF1428EBF0A65D58CD29981, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:44,475 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:44,475 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 323E88D7FAF1428EBF0A65D58CD29981
2016-01-14 17:16:44,475 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 323E88D7FAF1428EBF0A65D58CD29981
2016-01-14 17:16:44,477 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:44,477 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 323E88D7FAF1428EBF0A65D58CD29981
2016-01-14 17:16:44,477 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 323E88D7FAF1428EBF0A65D58CD29981
2016-01-14 17:16:44,477 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B29662284148463C9012B2AB6C5AFC8F, ordinal: 0
2016-01-14 17:16:44,477 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B29662284148463C9012B2AB6C5AFC8F, mem on start (mb), free: 599, total: 684, max: 684
2016-01-14 17:16:44,478 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:44,478 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:44,478 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:44,478 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:44,478 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:44,478 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:44,478 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:44,479 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:44,479 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:44,479 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 717225984, TotalAvailable: 480541409, TotalRequested/TotalJVMHeap:0.15
2016-01-14 17:16:44,479 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, C2449BC657DB4D4BBE29CCAD5BCD9703, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:44,479 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 323E88D7FAF1428EBF0A65D58CD29981, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:44,479 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:44,480 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 323E88D7FAF1428EBF0A65D58CD29981 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:44,480 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:44,480 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:44,480 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 323E88D7FAF1428EBF0A65D58CD29981
2016-01-14 17:16:44,480 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:44,480 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 323E88D7FAF1428EBF0A65D58CD29981
2016-01-14 17:16:44,481 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:44,481 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791804288_0001_1_00_000000_0
2016-01-14 17:16:44,481 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:44,481 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@43deacb4
2016-01-14 17:16:44,482 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@43deacb4
2016-01-14 17:16:44,482 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:44,482 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:44,484 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:44,484 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 323E88D7FAF1428EBF0A65D58CD29981
2016-01-14 17:16:44,484 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 57A0AAA5394C4C46B92348944658951E
2016-01-14 17:16:44,485 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 57A0AAA5394C4C46B92348944658951E
2016-01-14 17:16:44,485 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@736cbe48; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@6840a42b; comparator=cascading.tuple.hadoop.util.TupleComparator@6bda3885; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:44,485 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:44,486 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:44,486 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:44,500 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:44,500 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:44,500 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:44,500 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:44,500 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:44,501 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:44,501 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:44,501 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B29662284148463C9012B2AB6C5AFC8F, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:44,503 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791804288_0001_1_00_000000_0
2016-01-14 17:16:44,503 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B29662284148463C9012B2AB6C5AFC8F, mem on close (mb), free: 497, total: 684, max: 684
2016-01-14 17:16:44,504 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:44,504 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:44,504 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:44,504 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:44,506 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:44 Completed running task attempt: attempt_1452791804288_0001_1_00_000000_0
2016-01-14 17:16:44,508 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791804288_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:44,509 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=717225984, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791804501, Process_Duration=2, Process_End_Time=1452791804503, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:44,509 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791804288_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791804288_0001_1_00, eventCount=1
2016-01-14 17:16:44,509 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:44,509 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:44,509 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:44,510 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791804288_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B29662284148463C9012B2AB6C5AFC8F, taskAttemptId=attempt_1452791804288_0001_1_00_000000_0, startTime=1452791804467, finishTime=1452791804509, timeTaken=42, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=717225984, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791804501, Process_Duration=2, Process_End_Time=1452791804503, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:44,510 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791804288_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:44,510 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791804288_0001_00_000001
2016-01-14 17:16:44,510 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:44,510 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791804288_0001_1_00_000000_0
2016-01-14 17:16:44,510 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:44,510 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791804288_0001_1][Event:TASK_FINISHED]: vertexName=B29662284148463C9012B2AB6C5AFC8F, taskId=task_1452791804288_0001_1_00_000000, startTime=1452791804467, finishTime=1452791804510, timeTaken=43, status=SUCCEEDED, successfulAttemptID=attempt_1452791804288_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=717225984, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791804501, Process_Duration=2, Process_End_Time=1452791804503, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:44,511 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791804288_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:44,511 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791804288_0001_1_00 [B29662284148463C9012B2AB6C5AFC8F] : 1
2016-01-14 17:16:44,511 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791804288_0001_1_00 [B29662284148463C9012B2AB6C5AFC8F]
2016-01-14 17:16:44,511 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:44,511 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791804288_0001_1][Event:VERTEX_FINISHED]: vertexName=B29662284148463C9012B2AB6C5AFC8F, vertexId=vertex_1452791804288_0001_1_00, initRequestedTime=1452791804448, initedTime=1452791804456, startRequestedTime=1452791804458, startedTime=1452791804458, finishTime=1452791804511, timeTaken=53, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=717225984, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791804501, Process_Duration=2, Process_End_Time=1452791804503, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791804467, firstTasksToStart=[ task_1452791804288_0001_1_00_000000 ], lastTaskFinishTime=1452791804509, lastTasksToFinish=[ task_1452791804288_0001_1_00_000000 ], minTaskDuration=42, maxTaskDuration=42, avgTaskDuration=42.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791804288_0001_1_00_000000 ], longestDurationTasks=[ task_1452791804288_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:44,512 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791804288_0001_1_00 [B29662284148463C9012B2AB6C5AFC8F] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:44,512 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791804288_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:44,512 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791804288_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791804288_0001_00_000001, stoppedTime=1452791804512, exitStatus=0
2016-01-14 17:16:44,512 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791804288_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:44,512 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791804288_0001_1_01 [C2449BC657DB4D4BBE29CCAD5BCD9703] attempt: attempt_1452791804288_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:44,512 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: C2449BC657DB4D4BBE29CCAD5BCD9703
2016-01-14 17:16:44,512 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791804288_0001_1_01 [C2449BC657DB4D4BBE29CCAD5BCD9703]
2016-01-14 17:16:44,513 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791804288_0001_1_00 [B29662284148463C9012B2AB6C5AFC8F] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:44,512 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:44,513 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791804288_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:44,513 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:44,513 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791804288_0001_1][Event:TASK_STARTED]: vertexName=C2449BC657DB4D4BBE29CCAD5BCD9703, taskId=task_1452791804288_0001_1_01_000000, scheduledTime=1452791804513, launchTime=1452791804513
2016-01-14 17:16:44,513 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791804288_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:44,514 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791804288_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:44,514 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791804288_0001_1_01 [C2449BC657DB4D4BBE29CCAD5BCD9703]
2016-01-14 17:16:44,514 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [150CC68B228D48C7A180A99C61A5ABEC/A97404B827D048BA817E079B533E6C3F] every/(1/1) ...tform/ALL_'field'_RESULTS, VertexName: C2449BC657DB4D4BBE29CCAD5BCD9703, VertexParallelism: 1, TaskAttemptID:attempt_1452791804288_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B29662284148463C9012B2AB6C5AFC8F, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=FEB5F4D4E2AD4E73B209982A332B34B2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:44,514 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791804288_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:44,514 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:44,515 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791804288_0001_00_000002
2016-01-14 17:16:44,515 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791804288_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:44,516 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791804288_0001_00_000002
2016-01-14 17:16:44,517 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:44,517 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791804288_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:44,517 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791804288_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791804288_0001_00_000002, launchTime=1452791804517
2016-01-14 17:16:44,517 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:44,518 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791804288_0001_00_000002
2016-01-14 17:16:44,518 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791804288_0001_1_01_000000_0] to container: [container_1452791804288_0001_00_000002]
2016-01-14 17:16:44,518 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791804288_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:44,518 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791804288_0001_1_01_000000_0] started. Is using containerId: [container_1452791804288_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:44,519 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791804288_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C2449BC657DB4D4BBE29CCAD5BCD9703, taskAttemptId=attempt_1452791804288_0001_1_01_000000_0, startTime=1452791804518, containerId=container_1452791804288_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791804288_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:44,519 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:44,519 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791804288_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:44,519 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791804288_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:44,519 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791804288_0001_00_000002 given task: attempt_1452791804288_0001_1_01_000000_0
2016-01-14 17:16:44,519 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791804288_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791804288_0001_1_01_000000_0
2016-01-14 17:16:44 Starting to run new task attempt: attempt_1452791804288_0001_1_01_000000_0
2016-01-14 17:16:44,519 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791804288_0001_1_01_000000_0
2016-01-14 17:16:44,519 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:44,520 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:44,520 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:44,520 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:44,520 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:44,520 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:44,520 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [150CC68B228D48C7A180A99C61A5ABEC/A97404B827D048BA817E079B533E6C3F] every/(1/1) ...tform/ALL_'field'_RESULTS, VertexName: C2449BC657DB4D4BBE29CCAD5BCD9703, VertexParallelism: 1, TaskAttemptID:attempt_1452791804288_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B29662284148463C9012B2AB6C5AFC8F, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=FEB5F4D4E2AD4E73B209982A332B34B2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:44,520 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=717225984
2016-01-14 17:16:44,520 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791804288_0001_1_01_000000_0
2016-01-14 17:16:44,520 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:44,521 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791804288_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:44,521 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:44,521 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791804288_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:44,521 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791804288_0001_00_000001, Exiting
2016-01-14 17:16:44,521 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791804288_0001_00_000001
2016-01-14 17:16:44,521 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791804288_0001_00_000001
2016-01-14 17:16:44,521 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B29662284148463C9012B2AB6C5AFC8F, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:44,521 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791804288_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:44,521 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:44,522 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791804288_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:44,521 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:44,523 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B29662284148463C9012B2AB6C5AFC8F
2016-01-14 17:16:44,524 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 645503360, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:44,524 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:44,524 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 645503360, type: INPUT, componentVertexName: B29662284148463C9012B2AB6C5AFC8F
2016-01-14 17:16:44,524 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:44,524 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791804288_0001_00_000001 completed successfully
2016-01-14 17:16:44,524 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B29662284148463C9012B2AB6C5AFC8F
2016-01-14 17:16:44,525 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791804288_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:44,526 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=FEB5F4D4E2AD4E73B209982A332B34B2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:44,527 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:44,527 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: FEB5F4D4E2AD4E73B209982A332B34B2
2016-01-14 17:16:44,527 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:44,527 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: FEB5F4D4E2AD4E73B209982A332B34B2
2016-01-14 17:16:44,528 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C2449BC657DB4D4BBE29CCAD5BCD9703, ordinal: 1
2016-01-14 17:16:44,528 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C2449BC657DB4D4BBE29CCAD5BCD9703, mem on start (mb), free: 492, total: 684, max: 684
2016-01-14 17:16:44,528 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:44,529 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:44,529 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:44,534 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:44,534 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: FEB5F4D4E2AD4E73B209982A332B34B2
2016-01-14 17:16:44,534 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:44,534 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:44,534 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:44,535 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:44,535 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:44,535 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:44,535 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 645503360, TotalRequestedScaled: 5.958492553846154E8, TotalJVMHeap: 717225984, TotalAvailable: 480541409, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:44,535 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B29662284148463C9012B2AB6C5AFC8F, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=645503360, allocated=480541409
2016-01-14 17:16:44,535 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, FEB5F4D4E2AD4E73B209982A332B34B2, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:44,536 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:44,536 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B29662284148463C9012B2AB6C5AFC8F being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:44,536 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B29662284148463C9012B2AB6C5AFC8F
2016-01-14 17:16:44,536 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:44,536 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:44,536 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:44,536 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:44,537 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: B29662284148463C9012B2AB6C5AFC8F with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:44,537 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=645503360, postMergeMem=0, RuntimeTotalAvailable=480541409. Updated to: ShuffleMem=480541409, postMergeMem: 0
2016-01-14 17:16:44,537 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=480541409, maxSingleShuffleLimit=120135352, mergeThreshold=432487264, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:44,537 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:44,538 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B29662284148463C9012B2AB6C5AFC8F
2016-01-14 17:16:44,538 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:44,539 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791804288_0001_1_01_000000_0
2016-01-14 17:16:44,540 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791804288_0001_1_00_000000_0_10242, runDuration: 34943, ]
2016-01-14 17:16:44,541 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791804288_0001_1_00_000000_0_10242] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791804541, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 17:16:44,541 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:44,541 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : B29662284148463C9012B2AB6C5AFC8F
2016-01-14 17:16:44,541 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 57A0AAA5394C4C46B92348944658951E
2016-01-14 17:16:44,542 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_RESULTS"], id: FEB5F4D4E2AD4E73B209982A332B34B2
2016-01-14 17:16:44,542 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_RESULTS"]
2016-01-14 17:16:44,542 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 57A0AAA5394C4C46B92348944658951E, for 1 inputs
2016-01-14 17:16:44,543 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:44,543 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [B29662284148463C9012B2AB6C5AFC8F] #41 in 3ms
2016-01-14 17:16:44,543 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [B29662284148463C9012B2AB6C5AFC8F] #41, status:false, isInterrupted:false
2016-01-14 17:16:44,543 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:44,544 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:44,544 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:44,544 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:44,544 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:44,544 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : B29662284148463C9012B2AB6C5AFC8F
2016-01-14 17:16:44,544 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:44,544 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C2449BC657DB4D4BBE29CCAD5BCD9703, all 1 inputs ready in: 00:00:00.002
2016-01-14 17:16:44,545 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@6650c94a; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@41cf4dc0
2016-01-14 17:16:44,547 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:44,547 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:44,548 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791804288_0001_1_01_000000_0
2016-01-14 17:16:44,548 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791804288_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:44,548 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791804288_0001_1_01_000000_0
2016-01-14 17:16:44,548 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C2449BC657DB4D4BBE29CCAD5BCD9703, mem on close (mb), free: 488, total: 684, max: 684
2016-01-14 17:16:44,549 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: B29662284148463C9012B2AB6C5AFC8F
2016-01-14 17:16:44,549 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:44,549 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:44,551 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:44,551 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:44,551 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:44 Completed running task attempt: attempt_1452791804288_0001_1_01_000000_0
2016-01-14 17:16:44,552 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791804288_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:44,552 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=717225984, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791804545, Process_Duration=3, Process_End_Time=1452791804548, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 17:16:44,552 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:44,553 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:44,553 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791804288_0001_00_000002
2016-01-14 17:16:44,553 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:44,553 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791804288_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C2449BC657DB4D4BBE29CCAD5BCD9703, taskAttemptId=attempt_1452791804288_0001_1_01_000000_0, startTime=1452791804518, finishTime=1452791804552, timeTaken=34, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=717225984, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791804545, Process_Duration=3, Process_End_Time=1452791804548, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:44,553 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791804288_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:44,553 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791804288_0001_1_01_000000_0
2016-01-14 17:16:44,554 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:44,554 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791804288_0001_1][Event:TASK_FINISHED]: vertexName=C2449BC657DB4D4BBE29CCAD5BCD9703, taskId=task_1452791804288_0001_1_01_000000, startTime=1452791804518, finishTime=1452791804554, timeTaken=36, status=SUCCEEDED, successfulAttemptID=attempt_1452791804288_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=717225984, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791804545, Process_Duration=3, Process_End_Time=1452791804548, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:44,554 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791804288_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:44,554 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791804288_0001_1_01 [C2449BC657DB4D4BBE29CCAD5BCD9703] : 1
2016-01-14 17:16:44,554 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791804288_0001_1_01 [C2449BC657DB4D4BBE29CCAD5BCD9703]
2016-01-14 17:16:44,555 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:44,555 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:44,555 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791804288_0001_1][Event:VERTEX_FINISHED]: vertexName=C2449BC657DB4D4BBE29CCAD5BCD9703, vertexId=vertex_1452791804288_0001_1_01, initRequestedTime=1452791804449, initedTime=1452791804455, startRequestedTime=1452791804460, startedTime=1452791804460, finishTime=1452791804554, timeTaken=94, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=717225984, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791804545, Process_Duration=3, Process_End_Time=1452791804548, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452791804518, firstTasksToStart=[ task_1452791804288_0001_1_01_000000 ], lastTaskFinishTime=1452791804552, lastTasksToFinish=[ task_1452791804288_0001_1_01_000000 ], minTaskDuration=34, maxTaskDuration=34, avgTaskDuration=34.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791804288_0001_1_01_000000 ], longestDurationTasks=[ task_1452791804288_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:44,555 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791804288_0001_1_01 [C2449BC657DB4D4BBE29CCAD5BCD9703] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:44,555 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791804288_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:44,555 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791804288_0001_1_01 [C2449BC657DB4D4BBE29CCAD5BCD9703] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:44,555 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791804288_0001_1
2016-01-14 17:16:44,556 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791804288_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452791804288_0001_1
2016-01-14 17:16:44,556 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: FEB5F4D4E2AD4E73B209982A332B34B2 for vertex: vertex_1452791804288_0001_1_01 [C2449BC657DB4D4BBE29CCAD5BCD9703]
2016-01-14 17:16:44,563 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452791804288_0001_1_00 [B29662284148463C9012B2AB6C5AFC8F]
2016-01-14 17:16:44,564 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791804288_0001_1, queueSize=0
2016-01-14 17:16:44,565 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:44,565 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791804288_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791804288_0001_1, startTime=1452791804448, finishTime=1452791804563, timeTaken=115, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=465, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1434451968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905583609046, Process_Duration=5, Process_End_Time=2905583609051, Read_Duration=3, Tuples_Read=40, Tuples_Written=40, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 17:16:44,565 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791804288_0001_1 finished with state: SUCCEEDED
2016-01-14 17:16:44,566 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791804288_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:44,566 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791804288_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791804288_0001_00_000002, stoppedTime=1452791804566, exitStatus=0
2016-01-14 17:16:44,566 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:44,566 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791804288_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:44 Completed Dag: dag_1452791804288_0001_1
2016-01-14 17:16:44,567 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791804288_0001_1, dagState=SUCCEEDED
2016-01-14 17:16:44,568 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791804288_0001_1_post
2016-01-14 17:16:44,568 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:44,568 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:44,569 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791804288_0001_00_000002. Relying on regular task shutdown for it to end
2016-01-14 17:16:44,569 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791804288_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:44,572 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...tform/ALL_'field'_RESULTS, applicationId=application_1452791804288_0001
2016-01-14 17:16:44,574 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:44,574 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:44,574 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:44,574 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:44,574 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:44,574 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:44,574 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:44,575 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:44,575 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:44,575 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:44,575 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:44,575 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:44,576 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:44,576 INFO  task.TezChild (TezChild.java:run(198)) - Interrupted while waiting for new work for container container_1452791804288_0001_00_000002
2016-01-14 17:16:44,576 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791804288_0001_00_000002
2016-01-14 17:16:44,576 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791804288_0001_00_000002
2016-01-14 17:16:44,576 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791804288_0001_00_000002 = ContainerExecutionResult{exitStatus=INTERRUPTED, throwable=java.lang.InterruptedException, errorMessage='Interrupted while waiting for new work'}
2016-01-14 17:16:44,576 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1452791804288_0001_00_000002 completed but with errors
2016-01-14 17:16:44,577 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1452791804288_0001_00_000002 exited with diagnostics set to Interrupted while waiting for new work
2016-01-14 17:16:44,577 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791804288_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:44,582 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/150CC68B228D48C7A180A99C61A5ABEC/A97404B827D048BA817E079B533E6C3F/.tez/application_1452791804288_0001
2016-01-14 17:16:44,583 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.321
2016-01-14 17:16:44,643 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_'field'_RESULTS
2016-01-14 17:16:44,663 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:44,663 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:44,691 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.007
2016-01-14 17:16:44,701 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.017
2016-01-14 17:16:44,702 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:44,702 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:44,702 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:44,716 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:44,721 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_ALL
2016-01-14 17:16:44,776 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:44,776 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:44,776 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_ALL"]
2016-01-14 17:16:44,776 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:44,776 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:44,776 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:44,777 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...yplatform/ALL_'field'_ALL
2016-01-14 17:16:44,777 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:44,788 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:44,792 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:44,792 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:44,792 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:44,800 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/9CB7510594714803A429C2D9142A29E4/B1BDB2C335C74A5E80185AC1FDEFC4F6/.tez/application_1452791804792_0001 doesn't exist and is created
2016-01-14 17:16:44,810 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 17:16:44,810 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 17:16:44,817 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:44,817 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:44,817 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/9CB7510594714803A429C2D9142A29E4/B1BDB2C335C74A5E80185AC1FDEFC4F6/.tez/application_1452791804792_0001
2016-01-14 17:16:44,819 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791804792_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:44,829 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:44,829 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:44,830 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:44,830 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:44,830 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/9CB7510594714803A429C2D9142A29E4/B1BDB2C335C74A5E80185AC1FDEFC4F6/.tez/application_1452791804792_0001/localmode-log-dir
2016-01-14 17:16:44,830 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/9CB7510594714803A429C2D9142A29E4/B1BDB2C335C74A5E80185AC1FDEFC4F6/.tez/application_1452791804792_0001/localmode-log-dir/history.txt.appattempt_1452791804792_0001_000000, maxErrors=10
2016-01-14 17:16:44,831 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:44,831 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791804792_0001_000000, appSubmitTime=1452791804819, launchTime=1452791804819
2016-01-14 17:16:44,833 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:44,834 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:44,835 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:44,835 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:58823
2016-01-14 17:16:44,837 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:44,838 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:44,839 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791804792_0001_000000, startTime=1452791804839
2016-01-14 17:16:44,839 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:44,839 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:44,917 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:44,917 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:44,917 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...yplatform/ALL_'field'_ALL, applicationId=application_1452791804792_0001, dagName=[9CB7510594714803A429C2D9142A29E4/B1BDB2C335C74A5E80185AC1FDEFC4F6] every/(1/1) ...yplatform/ALL_'field'_ALL
2016-01-14 17:16:44,930 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [9CB7510594714803A429C2D9142A29E4/B1BDB2C335C74A5E80185AC1FDEFC4F6] every/(1/1) ...yplatform/ALL_'field'_ALL
2016-01-14 17:16:44,931 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791804792_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/9CB7510594714803A429C2D9142A29E4/B1BDB2C335C74A5E80185AC1FDEFC4F6/.tez/application_1452791804792_0001/localmode-log-dir/dag_1452791804792_0001_1.dot
2016-01-14 17:16:44,931 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/9CB7510594714803A429C2D9142A29E4/B1BDB2C335C74A5E80185AC1FDEFC4F6/.tez/application_1452791804792_0001/localmode-log-dir/dag_1452791804792_0001_1-tez-dag.pb.txt
2016-01-14 17:16:44,937 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791804792_0001_1
2016-01-14 17:16:44,937 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:44,937 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [9CB7510594714803A429C2D9142A29E4/B1BDB2C335C74A5E80185AC1FDEFC4F6] every/(1/1) ...yplatform/ALL_'field'_ALL
2016-01-14 17:16:44 Running Dag: dag_1452791804792_0001_1
2016-01-14 17:16:44,947 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791804792_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791804792_0001_1, submitTime=1452791804930
2016-01-14 17:16:44,947 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:44,948 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex FF1B0DD99B6D408BB4DE6F55E8E8D99A
2016-01-14 17:16:44,948 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:44,948 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791804792_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791804792_0001_1, initTime=1452791804947
2016-01-14 17:16:44,948 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791804792_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:44,948 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:44,949 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791804792_0001_1][Event:DAG_STARTED]: dagID=dag_1452791804792_0001_1, startTime=1452791804949
2016-01-14 17:16:44,949 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:44,949 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791804792_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:44,949 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...yplatform/ALL_'field'_ALL, applicationId=application_1452791804792_0001, dagName=[9CB7510594714803A429C2D9142A29E4/B1BDB2C335C74A5E80185AC1FDEFC4F6] every/(1/1) ...yplatform/ALL_'field'_ALL
2016-01-14 17:16:44,949 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:44,949 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: BDDB1958F4624394B3D892DBF8F59987 : {2A061C0D69174B809A66171C07CF4AE9={InputName=2A061C0D69174B809A66171C07CF4AE9}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:44,949 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 2A061C0D69174B809A66171C07CF4AE9, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:44,949 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791804792_0001_1_00 [BDDB1958F4624394B3D892DBF8F59987]
2016-01-14 17:16:44,950 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791804792_0001_1_00 [BDDB1958F4624394B3D892DBF8F59987]
2016-01-14 17:16:44,950 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791804792_0001_1_00 [BDDB1958F4624394B3D892DBF8F59987]
2016-01-14 17:16:44,950 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791804792_0001_1_00 [BDDB1958F4624394B3D892DBF8F59987]. Starting root input initializers: 1
2016-01-14 17:16:44,950 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791804792_0001_1_00 [BDDB1958F4624394B3D892DBF8F59987] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:44,950 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791804792_0001_1_01 [FF1B0DD99B6D408BB4DE6F55E8E8D99A]
2016-01-14 17:16:44,951 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 2A061C0D69174B809A66171C07CF4AE9 on vertex vertex_1452791804792_0001_1_00 [BDDB1958F4624394B3D892DBF8F59987]
2016-01-14 17:16:44,952 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:44,952 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 2A061C0D69174B809A66171C07CF4AE9 asking for 1 tasks. Headroom: 684 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:44,953 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:44,953 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:44,953 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:44,953 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791804792_0001_1_01 [FF1B0DD99B6D408BB4DE6F55E8E8D99A]
2016-01-14 17:16:44,953 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791804792_0001_1_01 [FF1B0DD99B6D408BB4DE6F55E8E8D99A]
2016-01-14 17:16:44,954 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791804792_0001_1_01 [FF1B0DD99B6D408BB4DE6F55E8E8D99A]
2016-01-14 17:16:44,954 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=0744DBD9B73D41D5BF9C74AC023F71FD, vertexId=vertex_1452791804792_0001_1_01 [FF1B0DD99B6D408BB4DE6F55E8E8D99A], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:44,954 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=0744DBD9B73D41D5BF9C74AC023F71FD, vertexId=vertex_1452791804792_0001_1_01 [FF1B0DD99B6D408BB4DE6F55E8E8D99A]
2016-01-14 17:16:44,955 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:44,955 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 2A061C0D69174B809A66171C07CF4AE9 on vertex vertex_1452791804792_0001_1_00 [BDDB1958F4624394B3D892DBF8F59987]
2016-01-14 17:16:44,956 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=0744DBD9B73D41D5BF9C74AC023F71FD, vertexName=FF1B0DD99B6D408BB4DE6F55E8E8D99A, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:44,956 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=0744DBD9B73D41D5BF9C74AC023F71FD, vertexId=vertex_1452791804792_0001_1_01 [FF1B0DD99B6D408BB4DE6F55E8E8D99A]
2016-01-14 17:16:44,957 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:44,957 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791804792_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FF1B0DD99B6D408BB4DE6F55E8E8D99A, vertexId=vertex_1452791804792_0001_1_01, initRequestedTime=1452791804950, initedTime=1452791804957, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:44,957 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791804792_0001_1_01 [FF1B0DD99B6D408BB4DE6F55E8E8D99A] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:44,957 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791804792_0001_1_00 [BDDB1958F4624394B3D892DBF8F59987] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:44,957 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {2A061C0D69174B809A66171C07CF4AE9=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:44,958 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791804792_0001_1_00 [BDDB1958F4624394B3D892DBF8F59987] parallelism set to 1
2016-01-14 17:16:44,958 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791804792_0001_1_00 [BDDB1958F4624394B3D892DBF8F59987] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:44,958 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791804792_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791804792_0001_1_00, eventCount=1
2016-01-14 17:16:44,958 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:44,959 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791804792_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BDDB1958F4624394B3D892DBF8F59987, vertexId=vertex_1452791804792_0001_1_00, initRequestedTime=1452791804949, initedTime=1452791804958, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:44,959 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791804792_0001_1_00 [BDDB1958F4624394B3D892DBF8F59987] with distanceFromRoot: 0
2016-01-14 17:16:44,959 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791804792_0001_1_00 [BDDB1958F4624394B3D892DBF8F59987] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:44,959 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in BDDB1958F4624394B3D892DBF8F59987
2016-01-14 17:16:44,959 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791804792_0001_1_00 [BDDB1958F4624394B3D892DBF8F59987]
2016-01-14 17:16:44,959 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:44,959 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791804792_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791804792_0001_1_00, startRequestedTime=1452791804959, startedTime=1452791804959
2016-01-14 17:16:44,959 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791804792_0001_1_00 [BDDB1958F4624394B3D892DBF8F59987] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:44,960 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:44,960 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791804792_0001_1][Event:TASK_STARTED]: vertexName=BDDB1958F4624394B3D892DBF8F59987, taskId=task_1452791804792_0001_1_00_000000, scheduledTime=1452791804960, launchTime=1452791804960
2016-01-14 17:16:44,960 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791804792_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:44,960 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791804792_0001_1_00 for vertex: vertex_1452791804792_0001_1_01 [FF1B0DD99B6D408BB4DE6F55E8E8D99A] numStartedSources: 1 numSources: 1
2016-01-14 17:16:44,960 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791804792_0001_1_01 [FF1B0DD99B6D408BB4DE6F55E8E8D99A] with distanceFromRoot: 1
2016-01-14 17:16:44,960 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791804792_0001_1_00 [BDDB1958F4624394B3D892DBF8F59987]
2016-01-14 17:16:44,960 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: BDDB1958F4624394B3D892DBF8F59987 in vertex: FF1B0DD99B6D408BB4DE6F55E8E8D99A
2016-01-14 17:16:44,961 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: FF1B0DD99B6D408BB4DE6F55E8E8D99A with 1 source tasks and 1 pending tasks
2016-01-14 17:16:44,961 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:44,961 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791804792_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791804792_0001_1_01, startRequestedTime=1452791804960, startedTime=1452791804960
2016-01-14 17:16:44,961 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791804792_0001_1_01 [FF1B0DD99B6D408BB4DE6F55E8E8D99A] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:44,961 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [9CB7510594714803A429C2D9142A29E4/B1BDB2C335C74A5E80185AC1FDEFC4F6] every/(1/1) ...yplatform/ALL_'field'_ALL, VertexName: BDDB1958F4624394B3D892DBF8F59987, VertexParallelism: 1, TaskAttemptID:attempt_1452791804792_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=2A061C0D69174B809A66171C07CF4AE9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FF1B0DD99B6D408BB4DE6F55E8E8D99A, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:44,961 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791804792_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:44,961 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:44,962 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:44,962 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:44,962 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:44,963 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791804792_0001_00_000001
2016-01-14 17:16:44,963 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791804792_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:44,964 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791804792_0001_00_000001
2016-01-14 17:16:44,964 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:44,965 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791804792_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:44,965 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791804792_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791804792_0001_00_000001, launchTime=1452791804965
2016-01-14 17:16:44,965 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:44,966 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791804792_0001_00_000001
2016-01-14 17:16:44,966 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791804792_0001_1_00_000000_0] to container: [container_1452791804792_0001_00_000001]
2016-01-14 17:16:44,966 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791804792_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:44,967 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791804792_0001_1_00_000000_0] started. Is using containerId: [container_1452791804792_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:44,967 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791804792_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BDDB1958F4624394B3D892DBF8F59987, taskAttemptId=attempt_1452791804792_0001_1_00_000000_0, startTime=1452791804966, containerId=container_1452791804792_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791804792_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:44,967 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791804792_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:44,967 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791804792_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:44,967 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:44,967 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791804792_0001_00_000001 given task: attempt_1452791804792_0001_1_00_000000_0
2016-01-14 17:16:44,968 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791804792_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791804792_0001_1_00_000000_0
2016-01-14 17:16:44 Starting to run new task attempt: attempt_1452791804792_0001_1_00_000000_0
2016-01-14 17:16:44,968 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791804792_0001_1_00_000000_0
2016-01-14 17:16:44,968 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:44,968 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:44,968 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:44,968 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:44,969 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:44,969 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:44,969 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [9CB7510594714803A429C2D9142A29E4/B1BDB2C335C74A5E80185AC1FDEFC4F6] every/(1/1) ...yplatform/ALL_'field'_ALL, VertexName: BDDB1958F4624394B3D892DBF8F59987, VertexParallelism: 1, TaskAttemptID:attempt_1452791804792_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=2A061C0D69174B809A66171C07CF4AE9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FF1B0DD99B6D408BB4DE6F55E8E8D99A, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:44,969 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=717225984
2016-01-14 17:16:44,969 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791804792_0001, with dag id: dag_1452791804792_0001_1
2016-01-14 17:16:44,970 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791804792_0001_1_00_000000_0
2016-01-14 17:16:44,970 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:44,970 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:44,970 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:44,970 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=FF1B0DD99B6D408BB4DE6F55E8E8D99A, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:44,971 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:44,971 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: FF1B0DD99B6D408BB4DE6F55E8E8D99A
2016-01-14 17:16:44,971 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=2A061C0D69174B809A66171C07CF4AE9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:44,972 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:44,972 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 2A061C0D69174B809A66171C07CF4AE9
2016-01-14 17:16:44,972 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 2A061C0D69174B809A66171C07CF4AE9
2016-01-14 17:16:44,972 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:44,972 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: FF1B0DD99B6D408BB4DE6F55E8E8D99A
2016-01-14 17:16:44,972 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: FF1B0DD99B6D408BB4DE6F55E8E8D99A
2016-01-14 17:16:44,973 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:44,976 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:44,976 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 2A061C0D69174B809A66171C07CF4AE9
2016-01-14 17:16:44,976 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BDDB1958F4624394B3D892DBF8F59987, ordinal: 0
2016-01-14 17:16:44,976 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BDDB1958F4624394B3D892DBF8F59987, mem on start (mb), free: 586, total: 684, max: 684
2016-01-14 17:16:44,976 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:44,976 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:44,976 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:44,976 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 2A061C0D69174B809A66171C07CF4AE9
2016-01-14 17:16:44,977 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791804792_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:44,977 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:44,977 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:44,977 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:44,977 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:44,977 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:44,978 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:44,978 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 717225984, TotalAvailable: 480541409, TotalRequested/TotalJVMHeap:0.15
2016-01-14 17:16:44,978 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 2A061C0D69174B809A66171C07CF4AE9, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:44,978 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, FF1B0DD99B6D408BB4DE6F55E8E8D99A, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:44,978 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:44,979 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 2A061C0D69174B809A66171C07CF4AE9 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:44,979 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:44,979 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:44,979 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 2A061C0D69174B809A66171C07CF4AE9
2016-01-14 17:16:44,979 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:44,979 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 2A061C0D69174B809A66171C07CF4AE9
2016-01-14 17:16:44,980 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:44,980 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791804792_0001_1_00_000000_0
2016-01-14 17:16:44,980 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:44,981 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@67f56c9a
2016-01-14 17:16:44,981 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@67f56c9a
2016-01-14 17:16:44,981 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:44,982 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:44,983 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:44,983 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 2A061C0D69174B809A66171C07CF4AE9
2016-01-14 17:16:44,983 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: C88FC42F0C874B2583B50BB3812B6937
2016-01-14 17:16:44,983 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] C88FC42F0C874B2583B50BB3812B6937
2016-01-14 17:16:44,984 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@6ea814f5; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@5cf81994; comparator=cascading.tuple.hadoop.util.TupleComparator@2457733b; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:44,984 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:44,984 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:44,984 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:44,997 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:44,997 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:44,997 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:44,997 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:44,997 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:44,998 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:44,998 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:44,998 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BDDB1958F4624394B3D892DBF8F59987, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:45,000 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791804792_0001_1_00_000000_0
2016-01-14 17:16:45,000 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BDDB1958F4624394B3D892DBF8F59987, mem on close (mb), free: 483, total: 684, max: 684
2016-01-14 17:16:45,001 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:45,001 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:45,001 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:45,001 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:45,004 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:45 Completed running task attempt: attempt_1452791804792_0001_1_00_000000_0
2016-01-14 17:16:45,010 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791804792_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:45,010 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791804792_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791804792_0001_1_00, eventCount=1
2016-01-14 17:16:45,010 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=717225984, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791804998, Process_Duration=2, Process_End_Time=1452791805000, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:45,011 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791804792_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BDDB1958F4624394B3D892DBF8F59987, taskAttemptId=attempt_1452791804792_0001_1_00_000000_0, startTime=1452791804966, finishTime=1452791805010, timeTaken=44, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=717225984, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791804998, Process_Duration=2, Process_End_Time=1452791805000, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:45,011 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791804792_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:45,011 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:45,011 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791804792_0001_1_00_000000_0
2016-01-14 17:16:45,011 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:45,011 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:45,011 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791804792_0001_00_000001
2016-01-14 17:16:45,011 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791804792_0001_1][Event:TASK_FINISHED]: vertexName=BDDB1958F4624394B3D892DBF8F59987, taskId=task_1452791804792_0001_1_00_000000, startTime=1452791804966, finishTime=1452791805011, timeTaken=45, status=SUCCEEDED, successfulAttemptID=attempt_1452791804792_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=717225984, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791804998, Process_Duration=2, Process_End_Time=1452791805000, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:45,011 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791804792_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:45,012 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791804792_0001_1_00 [BDDB1958F4624394B3D892DBF8F59987] : 1
2016-01-14 17:16:45,012 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791804792_0001_1_00 [BDDB1958F4624394B3D892DBF8F59987]
2016-01-14 17:16:45,012 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791804792_0001_1][Event:VERTEX_FINISHED]: vertexName=BDDB1958F4624394B3D892DBF8F59987, vertexId=vertex_1452791804792_0001_1_00, initRequestedTime=1452791804949, initedTime=1452791804958, startRequestedTime=1452791804959, startedTime=1452791804959, finishTime=1452791805012, timeTaken=53, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=717225984, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791804998, Process_Duration=2, Process_End_Time=1452791805000, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791804966, firstTasksToStart=[ task_1452791804792_0001_1_00_000000 ], lastTaskFinishTime=1452791805010, lastTasksToFinish=[ task_1452791804792_0001_1_00_000000 ], minTaskDuration=44, maxTaskDuration=44, avgTaskDuration=44.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791804792_0001_1_00_000000 ], longestDurationTasks=[ task_1452791804792_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:45,012 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791804792_0001_1_00 [BDDB1958F4624394B3D892DBF8F59987] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:45,012 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791804792_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:45,012 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791804792_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791804792_0001_00_000001, stoppedTime=1452791805012, exitStatus=0
2016-01-14 17:16:45,012 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791804792_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:45,012 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791804792_0001_1_01 [FF1B0DD99B6D408BB4DE6F55E8E8D99A] attempt: attempt_1452791804792_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:45,013 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: FF1B0DD99B6D408BB4DE6F55E8E8D99A
2016-01-14 17:16:45,013 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791804792_0001_1_01 [FF1B0DD99B6D408BB4DE6F55E8E8D99A]
2016-01-14 17:16:45,013 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791804792_0001_1_00 [BDDB1958F4624394B3D892DBF8F59987] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:45,013 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791804792_0001_1][Event:TASK_STARTED]: vertexName=FF1B0DD99B6D408BB4DE6F55E8E8D99A, taskId=task_1452791804792_0001_1_01_000000, scheduledTime=1452791805013, launchTime=1452791805013
2016-01-14 17:16:45,013 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791804792_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:45,013 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791804792_0001_1_01 [FF1B0DD99B6D408BB4DE6F55E8E8D99A]
2016-01-14 17:16:45,013 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [9CB7510594714803A429C2D9142A29E4/B1BDB2C335C74A5E80185AC1FDEFC4F6] every/(1/1) ...yplatform/ALL_'field'_ALL, VertexName: FF1B0DD99B6D408BB4DE6F55E8E8D99A, VertexParallelism: 1, TaskAttemptID:attempt_1452791804792_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BDDB1958F4624394B3D892DBF8F59987, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=0744DBD9B73D41D5BF9C74AC023F71FD, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:45,011 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:45,013 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791804792_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:45,014 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:45,014 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:45,014 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791804792_0001_00_000002
2016-01-14 17:16:45,013 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791804792_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:45,014 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791804792_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:45,014 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791804792_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:45,016 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791804792_0001_00_000002
2016-01-14 17:16:45,016 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:45,014 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:45,016 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791804792_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:45,017 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791804792_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791804792_0001_00_000002, launchTime=1452791805016
2016-01-14 17:16:45,017 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:45,017 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:45,017 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:45,017 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791804792_0001_00_000002
2016-01-14 17:16:45,017 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791804792_0001_1_01_000000_0] to container: [container_1452791804792_0001_00_000002]
2016-01-14 17:16:45,017 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791804792_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:45,018 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791804792_0001_1_01_000000_0] started. Is using containerId: [container_1452791804792_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:45,018 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791804792_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FF1B0DD99B6D408BB4DE6F55E8E8D99A, taskAttemptId=attempt_1452791804792_0001_1_01_000000_0, startTime=1452791805018, containerId=container_1452791804792_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791804792_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:45,018 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:45,018 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791804792_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:45,018 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791804792_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:45,018 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791804792_0001_00_000002 given task: attempt_1452791804792_0001_1_01_000000_0
2016-01-14 17:16:45,019 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791804792_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791804792_0001_1_01_000000_0
2016-01-14 17:16:45 Starting to run new task attempt: attempt_1452791804792_0001_1_01_000000_0
2016-01-14 17:16:45,019 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791804792_0001_1_01_000000_0
2016-01-14 17:16:45,019 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:45,019 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:45,019 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:45,019 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:45,019 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:45,020 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:45,020 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [9CB7510594714803A429C2D9142A29E4/B1BDB2C335C74A5E80185AC1FDEFC4F6] every/(1/1) ...yplatform/ALL_'field'_ALL, VertexName: FF1B0DD99B6D408BB4DE6F55E8E8D99A, VertexParallelism: 1, TaskAttemptID:attempt_1452791804792_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BDDB1958F4624394B3D892DBF8F59987, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=0744DBD9B73D41D5BF9C74AC023F71FD, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:45,020 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=717225984
2016-01-14 17:16:45,020 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791804792_0001_1_01_000000_0
2016-01-14 17:16:45,020 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:45,020 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:45,020 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791804792_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:45,021 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:45,021 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=BDDB1958F4624394B3D892DBF8F59987, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:45,021 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:45,021 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: BDDB1958F4624394B3D892DBF8F59987
2016-01-14 17:16:45,022 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791804792_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:45,022 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791804792_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:45,022 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791804792_0001_00_000001, Exiting
2016-01-14 17:16:45,022 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791804792_0001_00_000001
2016-01-14 17:16:45,022 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791804792_0001_00_000001
2016-01-14 17:16:45,023 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 645503360, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:45,023 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:45,023 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 645503360, type: INPUT, componentVertexName: BDDB1958F4624394B3D892DBF8F59987
2016-01-14 17:16:45,023 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: BDDB1958F4624394B3D892DBF8F59987
2016-01-14 17:16:45,023 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791804792_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:45,024 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791804792_0001_00_000001 completed successfully
2016-01-14 17:16:45,024 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:45,024 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791804792_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:45,025 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=0744DBD9B73D41D5BF9C74AC023F71FD, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:45,025 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:45,025 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 0744DBD9B73D41D5BF9C74AC023F71FD
2016-01-14 17:16:45,025 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:45,025 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 0744DBD9B73D41D5BF9C74AC023F71FD
2016-01-14 17:16:45,028 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FF1B0DD99B6D408BB4DE6F55E8E8D99A, ordinal: 1
2016-01-14 17:16:45,028 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FF1B0DD99B6D408BB4DE6F55E8E8D99A, mem on start (mb), free: 478, total: 684, max: 684
2016-01-14 17:16:45,028 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:45,028 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:45,028 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:45,033 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:45,033 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 0744DBD9B73D41D5BF9C74AC023F71FD
2016-01-14 17:16:45,033 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:45,033 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:45,034 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:45,034 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:45,034 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:45,034 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:45,034 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 645503360, TotalRequestedScaled: 5.958492553846154E8, TotalJVMHeap: 717225984, TotalAvailable: 480541409, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:45,034 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, BDDB1958F4624394B3D892DBF8F59987, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=645503360, allocated=480541409
2016-01-14 17:16:45,034 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 0744DBD9B73D41D5BF9C74AC023F71FD, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:45,034 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:45,034 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: BDDB1958F4624394B3D892DBF8F59987 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:45,034 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: BDDB1958F4624394B3D892DBF8F59987
2016-01-14 17:16:45,035 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:45,035 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:45,035 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:45,035 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:45,035 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: BDDB1958F4624394B3D892DBF8F59987 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:45,035 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=645503360, postMergeMem=0, RuntimeTotalAvailable=480541409. Updated to: ShuffleMem=480541409, postMergeMem: 0
2016-01-14 17:16:45,035 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=480541409, maxSingleShuffleLimit=120135352, mergeThreshold=432487264, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:45,036 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:45,036 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: BDDB1958F4624394B3D892DBF8F59987
2016-01-14 17:16:45,036 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:45,036 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791804792_0001_1_01_000000_0
2016-01-14 17:16:45,037 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791804792_0001_1_00_000000_0_10248, runDuration: 36289, ]
2016-01-14 17:16:45,038 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791804792_0001_1_00_000000_0_10248] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791805038, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 17:16:45,038 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : BDDB1958F4624394B3D892DBF8F59987
2016-01-14 17:16:45,038 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:45,038 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [BDDB1958F4624394B3D892DBF8F59987] #42 in 1ms
2016-01-14 17:16:45,039 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [BDDB1958F4624394B3D892DBF8F59987] #42, status:false, isInterrupted:false
2016-01-14 17:16:45,039 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:45,039 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:45,039 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:45,039 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:45,040 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:45,040 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : BDDB1958F4624394B3D892DBF8F59987
2016-01-14 17:16:45,040 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:45,040 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:45,041 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: C88FC42F0C874B2583B50BB3812B6937
2016-01-14 17:16:45,041 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_ALL"], id: 0744DBD9B73D41D5BF9C74AC023F71FD
2016-01-14 17:16:45,041 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_ALL"]
2016-01-14 17:16:45,042 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] C88FC42F0C874B2583B50BB3812B6937, for 1 inputs
2016-01-14 17:16:45,042 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FF1B0DD99B6D408BB4DE6F55E8E8D99A, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:45,042 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@396df0c7; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@60dab645
2016-01-14 17:16:45,055 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:45,056 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:45,056 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791804792_0001_1_01_000000_0
2016-01-14 17:16:45,056 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791804792_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:45,057 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791804792_0001_1_01_000000_0
2016-01-14 17:16:45,057 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FF1B0DD99B6D408BB4DE6F55E8E8D99A, mem on close (mb), free: 473, total: 684, max: 684
2016-01-14 17:16:45,057 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: BDDB1958F4624394B3D892DBF8F59987
2016-01-14 17:16:45,057 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:45,057 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:45,057 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:45,057 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:45,057 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:45 Completed running task attempt: attempt_1452791804792_0001_1_01_000000_0
2016-01-14 17:16:45,058 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791804792_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:45,059 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791804792_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FF1B0DD99B6D408BB4DE6F55E8E8D99A, taskAttemptId=attempt_1452791804792_0001_1_01_000000_0, startTime=1452791805018, finishTime=1452791805058, timeTaken=40, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=717225984, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=3, MERGE_PHASE_TIME=5, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791805042, Process_Duration=15, Process_End_Time=1452791805057, Read_Duration=4, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:45,059 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791804792_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:45,059 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:45,059 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:45,059 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791804792_0001_1_01_000000_0
2016-01-14 17:16:45,059 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:45,059 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791804792_0001_1][Event:TASK_FINISHED]: vertexName=FF1B0DD99B6D408BB4DE6F55E8E8D99A, taskId=task_1452791804792_0001_1_01_000000, startTime=1452791805018, finishTime=1452791805059, timeTaken=41, status=SUCCEEDED, successfulAttemptID=attempt_1452791804792_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=717225984, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=3, MERGE_PHASE_TIME=5, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791805042, Process_Duration=15, Process_End_Time=1452791805057, Read_Duration=4, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:45,060 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791804792_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:45,060 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791804792_0001_1_01 [FF1B0DD99B6D408BB4DE6F55E8E8D99A] : 1
2016-01-14 17:16:45,060 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791804792_0001_1_01 [FF1B0DD99B6D408BB4DE6F55E8E8D99A]
2016-01-14 17:16:45,060 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:45,060 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791804792_0001_1][Event:VERTEX_FINISHED]: vertexName=FF1B0DD99B6D408BB4DE6F55E8E8D99A, vertexId=vertex_1452791804792_0001_1_01, initRequestedTime=1452791804950, initedTime=1452791804957, startRequestedTime=1452791804960, startedTime=1452791804960, finishTime=1452791805060, timeTaken=100, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=717225984, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=3, MERGE_PHASE_TIME=5, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791805042, Process_Duration=15, Process_End_Time=1452791805057, Read_Duration=4, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452791805018, firstTasksToStart=[ task_1452791804792_0001_1_01_000000 ], lastTaskFinishTime=1452791805058, lastTasksToFinish=[ task_1452791804792_0001_1_01_000000 ], minTaskDuration=40, maxTaskDuration=40, avgTaskDuration=40.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791804792_0001_1_01_000000 ], longestDurationTasks=[ task_1452791804792_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:45,060 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791804792_0001_1_01 [FF1B0DD99B6D408BB4DE6F55E8E8D99A] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:45,061 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791804792_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:45,061 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791804792_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791804792_0001_00_000002, stoppedTime=1452791805061, exitStatus=0
2016-01-14 17:16:45,061 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791804792_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:45,060 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=717225984, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=3, MERGE_PHASE_TIME=5, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791805042, Process_Duration=15, Process_End_Time=1452791805057, Read_Duration=4, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 17:16:45,061 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:45,061 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791804792_0001_1_01 [FF1B0DD99B6D408BB4DE6F55E8E8D99A] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:45,061 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791804792_0001_1
2016-01-14 17:16:45,061 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:45,062 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:45,061 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791804792_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452791804792_0001_1
2016-01-14 17:16:45,062 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452791804792_0001_1_00 [BDDB1958F4624394B3D892DBF8F59987]
2016-01-14 17:16:45,062 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791804792_0001_00_000002
2016-01-14 17:16:45,062 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 0744DBD9B73D41D5BF9C74AC023F71FD for vertex: vertex_1452791804792_0001_1_01 [FF1B0DD99B6D408BB4DE6F55E8E8D99A]
2016-01-14 17:16:45,063 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791804792_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 17:16:45,063 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791804792_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:45,064 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791804792_0001_00_000002, Exiting
2016-01-14 17:16:45,064 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791804792_0001_00_000002
2016-01-14 17:16:45,064 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791804792_0001_00_000002
2016-01-14 17:16:45,064 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791804792_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:45,064 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791804792_0001_00_000002 completed successfully
2016-01-14 17:16:45,070 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791804792_0001_1, queueSize=0
2016-01-14 17:16:45,071 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:45,071 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791804792_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791804792_0001_1, startTime=1452791804949, finishTime=1452791805069, timeTaken=120, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=571, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1434451968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=3, MERGE_PHASE_TIME=5, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905583610040, Process_Duration=17, Process_End_Time=2905583610057, Read_Duration=6, Tuples_Read=40, Tuples_Written=40, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 17:16:45,071 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791804792_0001_1 finished with state: SUCCEEDED
2016-01-14 17:16:45,071 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791804792_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:45,071 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791804792_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:16:45,071 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452791804792_0001_00_000002
2016-01-14 17:16:45 Completed Dag: dag_1452791804792_0001_1
2016-01-14 17:16:45,074 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791804792_0001_1, dagState=SUCCEEDED
2016-01-14 17:16:45,074 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791804792_0001_1_post
2016-01-14 17:16:45,074 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:45,074 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:45,077 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...yplatform/ALL_'field'_ALL, applicationId=application_1452791804792_0001
2016-01-14 17:16:45,079 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:45,079 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:45,079 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:45,079 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:45,079 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:45,079 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:45,080 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:45,080 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:45,080 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:45,080 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:45,080 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:45,080 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:45,080 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:45,084 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/9CB7510594714803A429C2D9142A29E4/B1BDB2C335C74A5E80185AC1FDEFC4F6/.tez/application_1452791804792_0001
2016-01-14 17:16:45,084 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.308
2016-01-14 17:16:45,146 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_'field'_ALL
2016-01-14 17:16:45,166 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:45,167 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:45,198 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.002
2016-01-14 17:16:45,200 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.004
2016-01-14 17:16:45,201 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 17:16:45,201 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 17:16:45,201 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_'field'_REPLACE
2016-01-14 17:16:45,225 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:45,225 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:45,253 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.007
2016-01-14 17:16:45,257 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.011
2016-01-14 17:16:45,257 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 17:16:45,258 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 17:16:45,258 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:45,258 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:45,258 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:45,270 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:45,274 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_SWAP
2016-01-14 17:16:45,316 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:45,316 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:45,316 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_SWAP"]
2016-01-14 17:16:45,316 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:45,317 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:45,317 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:45,317 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...platform/ALL_'field'_SWAP
2016-01-14 17:16:45,317 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:45,328 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:45,332 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:45,333 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:45,333 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:45,342 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/94693635D0844BACA8B4A6A360F6443A/AAA35E222C7F4D85A285413D94B2BCB9/.tez/application_1452791805333_0001 doesn't exist and is created
2016-01-14 17:16:45,357 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:45,358 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:45,358 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/94693635D0844BACA8B4A6A360F6443A/AAA35E222C7F4D85A285413D94B2BCB9/.tez/application_1452791805333_0001
2016-01-14 17:16:45,360 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791805333_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:45,370 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:45,370 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:45,371 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:45,371 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:45,371 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/94693635D0844BACA8B4A6A360F6443A/AAA35E222C7F4D85A285413D94B2BCB9/.tez/application_1452791805333_0001/localmode-log-dir
2016-01-14 17:16:45,371 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/94693635D0844BACA8B4A6A360F6443A/AAA35E222C7F4D85A285413D94B2BCB9/.tez/application_1452791805333_0001/localmode-log-dir/history.txt.appattempt_1452791805333_0001_000000, maxErrors=10
2016-01-14 17:16:45,371 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:45,371 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791805333_0001_000000, appSubmitTime=1452791805360, launchTime=1452791805360
2016-01-14 17:16:45,376 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:44656
2016-01-14 17:16:45,377 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:45,377 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:45,378 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:45,378 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:45,379 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:45,380 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791805333_0001_000000, startTime=1452791805380
2016-01-14 17:16:45,380 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:45,380 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:45,458 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:45,458 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:45,458 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...platform/ALL_'field'_SWAP, applicationId=application_1452791805333_0001, dagName=[94693635D0844BACA8B4A6A360F6443A/AAA35E222C7F4D85A285413D94B2BCB9] every/(1/1) ...platform/ALL_'field'_SWAP
2016-01-14 17:16:45,472 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [94693635D0844BACA8B4A6A360F6443A/AAA35E222C7F4D85A285413D94B2BCB9] every/(1/1) ...platform/ALL_'field'_SWAP
2016-01-14 17:16:45,473 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791805333_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/94693635D0844BACA8B4A6A360F6443A/AAA35E222C7F4D85A285413D94B2BCB9/.tez/application_1452791805333_0001/localmode-log-dir/dag_1452791805333_0001_1.dot
2016-01-14 17:16:45,473 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/94693635D0844BACA8B4A6A360F6443A/AAA35E222C7F4D85A285413D94B2BCB9/.tez/application_1452791805333_0001/localmode-log-dir/dag_1452791805333_0001_1-tez-dag.pb.txt
2016-01-14 17:16:45,479 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791805333_0001_1
2016-01-14 17:16:45,479 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:45,479 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [94693635D0844BACA8B4A6A360F6443A/AAA35E222C7F4D85A285413D94B2BCB9] every/(1/1) ...platform/ALL_'field'_SWAP
2016-01-14 17:16:45 Running Dag: dag_1452791805333_0001_1
2016-01-14 17:16:45,490 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791805333_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791805333_0001_1, submitTime=1452791805472
2016-01-14 17:16:45,490 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:45,490 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex A29043E6066E4C5F9BA9802BF9D2E312
2016-01-14 17:16:45,490 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:45,491 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:45,491 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791805333_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791805333_0001_1, initTime=1452791805490
2016-01-14 17:16:45,491 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791805333_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:45,491 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791805333_0001_1][Event:DAG_STARTED]: dagID=dag_1452791805333_0001_1, startTime=1452791805491
2016-01-14 17:16:45,491 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:45,491 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791805333_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:45,491 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: F4D7C0EE2F92429B98DABF575DD70EAE : {7052E90654804D05AEE71FFE7AED50BE={InputName=7052E90654804D05AEE71FFE7AED50BE}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:45,491 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 7052E90654804D05AEE71FFE7AED50BE, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:45,491 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791805333_0001_1_00 [F4D7C0EE2F92429B98DABF575DD70EAE]
2016-01-14 17:16:45,492 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791805333_0001_1_00 [F4D7C0EE2F92429B98DABF575DD70EAE]
2016-01-14 17:16:45,492 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791805333_0001_1_00 [F4D7C0EE2F92429B98DABF575DD70EAE]
2016-01-14 17:16:45,492 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791805333_0001_1_00 [F4D7C0EE2F92429B98DABF575DD70EAE]. Starting root input initializers: 1
2016-01-14 17:16:45,492 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:45,492 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791805333_0001_1_00 [F4D7C0EE2F92429B98DABF575DD70EAE] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:45,492 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791805333_0001_1_01 [A29043E6066E4C5F9BA9802BF9D2E312]
2016-01-14 17:16:45,493 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...platform/ALL_'field'_SWAP, applicationId=application_1452791805333_0001, dagName=[94693635D0844BACA8B4A6A360F6443A/AAA35E222C7F4D85A285413D94B2BCB9] every/(1/1) ...platform/ALL_'field'_SWAP
2016-01-14 17:16:45,493 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 7052E90654804D05AEE71FFE7AED50BE on vertex vertex_1452791805333_0001_1_00 [F4D7C0EE2F92429B98DABF575DD70EAE]
2016-01-14 17:16:45,495 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:45,495 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:45,496 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791805333_0001_1_01 [A29043E6066E4C5F9BA9802BF9D2E312]
2016-01-14 17:16:45,496 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791805333_0001_1_01 [A29043E6066E4C5F9BA9802BF9D2E312]
2016-01-14 17:16:45,496 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791805333_0001_1_01 [A29043E6066E4C5F9BA9802BF9D2E312]
2016-01-14 17:16:45,496 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=5C74BEFFA61C4413986537C5C58590C4, vertexId=vertex_1452791805333_0001_1_01 [A29043E6066E4C5F9BA9802BF9D2E312], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:45,497 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=5C74BEFFA61C4413986537C5C58590C4, vertexId=vertex_1452791805333_0001_1_01 [A29043E6066E4C5F9BA9802BF9D2E312]
2016-01-14 17:16:45,496 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 7052E90654804D05AEE71FFE7AED50BE asking for 1 tasks. Headroom: 684 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:45,497 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:45,498 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:45,499 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=5C74BEFFA61C4413986537C5C58590C4, vertexName=A29043E6066E4C5F9BA9802BF9D2E312, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:45,499 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=5C74BEFFA61C4413986537C5C58590C4, vertexId=vertex_1452791805333_0001_1_01 [A29043E6066E4C5F9BA9802BF9D2E312]
2016-01-14 17:16:45,499 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:45,500 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 7052E90654804D05AEE71FFE7AED50BE on vertex vertex_1452791805333_0001_1_00 [F4D7C0EE2F92429B98DABF575DD70EAE]
2016-01-14 17:16:45,501 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:45,501 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791805333_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A29043E6066E4C5F9BA9802BF9D2E312, vertexId=vertex_1452791805333_0001_1_01, initRequestedTime=1452791805492, initedTime=1452791805500, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:45,501 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791805333_0001_1_01 [A29043E6066E4C5F9BA9802BF9D2E312] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:45,501 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791805333_0001_1_00 [F4D7C0EE2F92429B98DABF575DD70EAE] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:45,502 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {7052E90654804D05AEE71FFE7AED50BE=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:45,502 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791805333_0001_1_00 [F4D7C0EE2F92429B98DABF575DD70EAE] parallelism set to 1
2016-01-14 17:16:45,502 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791805333_0001_1_00 [F4D7C0EE2F92429B98DABF575DD70EAE] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:45,503 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791805333_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791805333_0001_1_00, eventCount=1
2016-01-14 17:16:45,503 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791805333_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F4D7C0EE2F92429B98DABF575DD70EAE, vertexId=vertex_1452791805333_0001_1_00, initRequestedTime=1452791805491, initedTime=1452791805503, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:45,504 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791805333_0001_1_00 [F4D7C0EE2F92429B98DABF575DD70EAE] with distanceFromRoot: 0
2016-01-14 17:16:45,503 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:45,504 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791805333_0001_1_00 [F4D7C0EE2F92429B98DABF575DD70EAE] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:45,504 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in F4D7C0EE2F92429B98DABF575DD70EAE
2016-01-14 17:16:45,504 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791805333_0001_1_00 [F4D7C0EE2F92429B98DABF575DD70EAE]
2016-01-14 17:16:45,505 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791805333_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791805333_0001_1_00, startRequestedTime=1452791805504, startedTime=1452791805504
2016-01-14 17:16:45,505 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:45,505 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791805333_0001_1_00 [F4D7C0EE2F92429B98DABF575DD70EAE] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:45,506 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:45,506 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791805333_0001_1][Event:TASK_STARTED]: vertexName=F4D7C0EE2F92429B98DABF575DD70EAE, taskId=task_1452791805333_0001_1_00_000000, scheduledTime=1452791805505, launchTime=1452791805505
2016-01-14 17:16:45,506 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791805333_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:45,506 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791805333_0001_1_00 for vertex: vertex_1452791805333_0001_1_01 [A29043E6066E4C5F9BA9802BF9D2E312] numStartedSources: 1 numSources: 1
2016-01-14 17:16:45,507 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791805333_0001_1_01 [A29043E6066E4C5F9BA9802BF9D2E312] with distanceFromRoot: 1
2016-01-14 17:16:45,507 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791805333_0001_1_00 [F4D7C0EE2F92429B98DABF575DD70EAE]
2016-01-14 17:16:45,507 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: F4D7C0EE2F92429B98DABF575DD70EAE in vertex: A29043E6066E4C5F9BA9802BF9D2E312
2016-01-14 17:16:45,507 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: A29043E6066E4C5F9BA9802BF9D2E312 with 1 source tasks and 1 pending tasks
2016-01-14 17:16:45,508 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791805333_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791805333_0001_1_01, startRequestedTime=1452791805507, startedTime=1452791805507
2016-01-14 17:16:45,508 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:45,508 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791805333_0001_1_01 [A29043E6066E4C5F9BA9802BF9D2E312] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:45,508 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [94693635D0844BACA8B4A6A360F6443A/AAA35E222C7F4D85A285413D94B2BCB9] every/(1/1) ...platform/ALL_'field'_SWAP, VertexName: F4D7C0EE2F92429B98DABF575DD70EAE, VertexParallelism: 1, TaskAttemptID:attempt_1452791805333_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=7052E90654804D05AEE71FFE7AED50BE, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A29043E6066E4C5F9BA9802BF9D2E312, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:45,509 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791805333_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:45,509 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:45,510 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:45,510 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:45,510 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:45,510 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791805333_0001_00_000001
2016-01-14 17:16:45,511 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791805333_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:45,512 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791805333_0001_00_000001
2016-01-14 17:16:45,512 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:45,512 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791805333_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:45,512 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:45,512 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791805333_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791805333_0001_00_000001, launchTime=1452791805512
2016-01-14 17:16:45,513 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791805333_0001, with dag id: dag_1452791805333_0001_1
2016-01-14 17:16:45,513 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791805333_0001_00_000001
2016-01-14 17:16:45,514 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791805333_0001_1_00_000000_0] to container: [container_1452791805333_0001_00_000001]
2016-01-14 17:16:45,514 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791805333_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:45,514 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791805333_0001_00_000001 given task: attempt_1452791805333_0001_1_00_000000_0
2016-01-14 17:16:45,515 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791805333_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791805333_0001_1_00_000000_0
2016-01-14 17:16:45,515 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791805333_0001_1_00_000000_0] started. Is using containerId: [container_1452791805333_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:45,515 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791805333_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F4D7C0EE2F92429B98DABF575DD70EAE, taskAttemptId=attempt_1452791805333_0001_1_00_000000_0, startTime=1452791805514, containerId=container_1452791805333_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791805333_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:45,516 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791805333_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:45,516 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:45,516 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791805333_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:45 Starting to run new task attempt: attempt_1452791805333_0001_1_00_000000_0
2016-01-14 17:16:45,516 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791805333_0001_1_00_000000_0
2016-01-14 17:16:45,516 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:45,517 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:45,517 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:45,517 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:45,517 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:45,517 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:45,517 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [94693635D0844BACA8B4A6A360F6443A/AAA35E222C7F4D85A285413D94B2BCB9] every/(1/1) ...platform/ALL_'field'_SWAP, VertexName: F4D7C0EE2F92429B98DABF575DD70EAE, VertexParallelism: 1, TaskAttemptID:attempt_1452791805333_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=7052E90654804D05AEE71FFE7AED50BE, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A29043E6066E4C5F9BA9802BF9D2E312, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:45,518 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=717225984
2016-01-14 17:16:45,519 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791805333_0001_1_00_000000_0
2016-01-14 17:16:45,519 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:45,519 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:45,519 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791805333_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:45,519 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=7052E90654804D05AEE71FFE7AED50BE, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:45,519 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:45,522 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:45,523 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A29043E6066E4C5F9BA9802BF9D2E312, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:45,523 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:45,523 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 7052E90654804D05AEE71FFE7AED50BE
2016-01-14 17:16:45,524 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 7052E90654804D05AEE71FFE7AED50BE
2016-01-14 17:16:45,524 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:45,525 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A29043E6066E4C5F9BA9802BF9D2E312
2016-01-14 17:16:45,526 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F4D7C0EE2F92429B98DABF575DD70EAE, ordinal: 0
2016-01-14 17:16:45,526 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F4D7C0EE2F92429B98DABF575DD70EAE, mem on start (mb), free: 556, total: 684, max: 684
2016-01-14 17:16:45,526 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:45,526 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:45,527 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:45,527 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: A29043E6066E4C5F9BA9802BF9D2E312
2016-01-14 17:16:45,528 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A29043E6066E4C5F9BA9802BF9D2E312
2016-01-14 17:16:45,528 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:45,528 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 7052E90654804D05AEE71FFE7AED50BE
2016-01-14 17:16:45,528 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 7052E90654804D05AEE71FFE7AED50BE
2016-01-14 17:16:45,528 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:45,529 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:45,529 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:45,529 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:45,529 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:45,529 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:45,530 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:45,530 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 717225984, TotalAvailable: 480541409, TotalRequested/TotalJVMHeap:0.15
2016-01-14 17:16:45,530 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 7052E90654804D05AEE71FFE7AED50BE, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:45,530 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A29043E6066E4C5F9BA9802BF9D2E312, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:45,530 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:45,530 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 7052E90654804D05AEE71FFE7AED50BE being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:45,531 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 7052E90654804D05AEE71FFE7AED50BE
2016-01-14 17:16:45,531 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:45,531 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 7052E90654804D05AEE71FFE7AED50BE
2016-01-14 17:16:45,531 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:45,531 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:45,531 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:45,532 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791805333_0001_1_00_000000_0
2016-01-14 17:16:45,532 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:45,532 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@2881d54e
2016-01-14 17:16:45,533 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@2881d54e
2016-01-14 17:16:45,533 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:45,533 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:45,535 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:45,535 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 7052E90654804D05AEE71FFE7AED50BE
2016-01-14 17:16:45,536 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 381C50FCDEE2484886D230FB43FD3F07
2016-01-14 17:16:45,536 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 381C50FCDEE2484886D230FB43FD3F07
2016-01-14 17:16:45,536 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@5f971825; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@38b2562a; comparator=cascading.tuple.hadoop.util.TupleComparator@4436fc7a; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:45,537 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:45,537 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:45,537 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:45,550 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:45,551 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:45,551 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:45,551 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:45,551 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:45,552 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:45,552 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:45,552 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F4D7C0EE2F92429B98DABF575DD70EAE, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:45,555 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791805333_0001_1_00_000000_0
2016-01-14 17:16:45,555 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F4D7C0EE2F92429B98DABF575DD70EAE, mem on close (mb), free: 454, total: 684, max: 684
2016-01-14 17:16:45,555 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:45,555 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:45,555 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:45,556 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:45,558 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:45 Completed running task attempt: attempt_1452791805333_0001_1_00_000000_0
2016-01-14 17:16:45,561 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791805333_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:45,561 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=717225984, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791805552, Process_Duration=3, Process_End_Time=1452791805555, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:45,562 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791805333_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791805333_0001_1_00, eventCount=1
2016-01-14 17:16:45,562 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791805333_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F4D7C0EE2F92429B98DABF575DD70EAE, taskAttemptId=attempt_1452791805333_0001_1_00_000000_0, startTime=1452791805514, finishTime=1452791805562, timeTaken=48, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=717225984, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791805552, Process_Duration=3, Process_End_Time=1452791805555, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:45,562 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791805333_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:45,562 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:45,562 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:45,562 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791805333_0001_1_00_000000_0
2016-01-14 17:16:45,563 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:45,563 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791805333_0001_1][Event:TASK_FINISHED]: vertexName=F4D7C0EE2F92429B98DABF575DD70EAE, taskId=task_1452791805333_0001_1_00_000000, startTime=1452791805514, finishTime=1452791805563, timeTaken=49, status=SUCCEEDED, successfulAttemptID=attempt_1452791805333_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=717225984, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791805552, Process_Duration=3, Process_End_Time=1452791805555, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:45,563 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791805333_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:45,563 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791805333_0001_1_00 [F4D7C0EE2F92429B98DABF575DD70EAE] : 1
2016-01-14 17:16:45,563 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791805333_0001_1_00 [F4D7C0EE2F92429B98DABF575DD70EAE]
2016-01-14 17:16:45,564 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:45,564 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791805333_0001_1][Event:VERTEX_FINISHED]: vertexName=F4D7C0EE2F92429B98DABF575DD70EAE, vertexId=vertex_1452791805333_0001_1_00, initRequestedTime=1452791805491, initedTime=1452791805503, startRequestedTime=1452791805504, startedTime=1452791805504, finishTime=1452791805563, timeTaken=59, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=717225984, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791805552, Process_Duration=3, Process_End_Time=1452791805555, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791805514, firstTasksToStart=[ task_1452791805333_0001_1_00_000000 ], lastTaskFinishTime=1452791805562, lastTasksToFinish=[ task_1452791805333_0001_1_00_000000 ], minTaskDuration=48, maxTaskDuration=48, avgTaskDuration=48.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791805333_0001_1_00_000000 ], longestDurationTasks=[ task_1452791805333_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:45,564 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791805333_0001_1_00 [F4D7C0EE2F92429B98DABF575DD70EAE] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:45,564 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791805333_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:45,564 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791805333_0001_1_01 [A29043E6066E4C5F9BA9802BF9D2E312] attempt: attempt_1452791805333_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:45,564 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: A29043E6066E4C5F9BA9802BF9D2E312
2016-01-14 17:16:45,564 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791805333_0001_1_01 [A29043E6066E4C5F9BA9802BF9D2E312]
2016-01-14 17:16:45,565 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791805333_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791805333_0001_00_000001, stoppedTime=1452791805565, exitStatus=0
2016-01-14 17:16:45,565 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:45,565 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791805333_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:45,565 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791805333_0001_1_00 [F4D7C0EE2F92429B98DABF575DD70EAE] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:45,565 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:45,565 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:45,565 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791805333_0001_1][Event:TASK_STARTED]: vertexName=A29043E6066E4C5F9BA9802BF9D2E312, taskId=task_1452791805333_0001_1_01_000000, scheduledTime=1452791805565, launchTime=1452791805565
2016-01-14 17:16:45,566 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791805333_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:45,565 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791805333_0001_00_000001
2016-01-14 17:16:45,565 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:45,566 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791805333_0001_1_01 [A29043E6066E4C5F9BA9802BF9D2E312]
2016-01-14 17:16:45,566 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791805333_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:45,566 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791805333_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:45,566 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791805333_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:45,566 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791805333_0001_00_000001, Exiting
2016-01-14 17:16:45,566 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791805333_0001_00_000001
2016-01-14 17:16:45,566 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791805333_0001_00_000001
2016-01-14 17:16:45,566 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [94693635D0844BACA8B4A6A360F6443A/AAA35E222C7F4D85A285413D94B2BCB9] every/(1/1) ...platform/ALL_'field'_SWAP, VertexName: A29043E6066E4C5F9BA9802BF9D2E312, VertexParallelism: 1, TaskAttemptID:attempt_1452791805333_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F4D7C0EE2F92429B98DABF575DD70EAE, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=5C74BEFFA61C4413986537C5C58590C4, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:45,567 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791805333_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:45,567 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791805333_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:45,567 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:45,568 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791805333_0001_00_000002
2016-01-14 17:16:45,568 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791805333_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:45,568 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791805333_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:45,568 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791805333_0001_00_000001 completed successfully
2016-01-14 17:16:45,569 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791805333_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:45,569 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791805333_0001_00_000002
2016-01-14 17:16:45,570 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 17:16:45,571 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791805333_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:45,572 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:45,572 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791805333_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791805333_0001_00_000002, launchTime=1452791805572
2016-01-14 17:16:45,572 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791805333_0001_00_000002
2016-01-14 17:16:45,573 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791805333_0001_1_01_000000_0] to container: [container_1452791805333_0001_00_000002]
2016-01-14 17:16:45,573 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791805333_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:45,573 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791805333_0001_00_000002 given task: attempt_1452791805333_0001_1_01_000000_0
2016-01-14 17:16:45,573 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791805333_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791805333_0001_1_01_000000_0
2016-01-14 17:16:45,574 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791805333_0001_1_01_000000_0] started. Is using containerId: [container_1452791805333_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:45,574 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791805333_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A29043E6066E4C5F9BA9802BF9D2E312, taskAttemptId=attempt_1452791805333_0001_1_01_000000_0, startTime=1452791805573, containerId=container_1452791805333_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791805333_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:45 Starting to run new task attempt: attempt_1452791805333_0001_1_01_000000_0
2016-01-14 17:16:45,574 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:45,574 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791805333_0001_1_01_000000_0
2016-01-14 17:16:45,574 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:45,574 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:45,574 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:45,574 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:45,574 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:45,575 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:45,575 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [94693635D0844BACA8B4A6A360F6443A/AAA35E222C7F4D85A285413D94B2BCB9] every/(1/1) ...platform/ALL_'field'_SWAP, VertexName: A29043E6066E4C5F9BA9802BF9D2E312, VertexParallelism: 1, TaskAttemptID:attempt_1452791805333_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F4D7C0EE2F92429B98DABF575DD70EAE, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=5C74BEFFA61C4413986537C5C58590C4, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:45,574 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791805333_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:45,575 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=717225984
2016-01-14 17:16:45,575 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791805333_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:45,575 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791805333_0001_1_01_000000_0
2016-01-14 17:16:45,575 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:45,576 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:45,576 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:45,576 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F4D7C0EE2F92429B98DABF575DD70EAE, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:45,576 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:45,576 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F4D7C0EE2F92429B98DABF575DD70EAE
2016-01-14 17:16:45,578 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 645503360, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:45,578 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:45,578 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 645503360, type: INPUT, componentVertexName: F4D7C0EE2F92429B98DABF575DD70EAE
2016-01-14 17:16:45,578 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F4D7C0EE2F92429B98DABF575DD70EAE
2016-01-14 17:16:45,579 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:45,579 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=5C74BEFFA61C4413986537C5C58590C4, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:45,579 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:45,580 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 5C74BEFFA61C4413986537C5C58590C4
2016-01-14 17:16:45,580 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:45,580 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 5C74BEFFA61C4413986537C5C58590C4
2016-01-14 17:16:45,580 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791805333_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:45,584 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A29043E6066E4C5F9BA9802BF9D2E312, ordinal: 1
2016-01-14 17:16:45,584 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A29043E6066E4C5F9BA9802BF9D2E312, mem on start (mb), free: 447, total: 684, max: 684
2016-01-14 17:16:45,584 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:45,584 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:45,585 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:45,589 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:45,589 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 5C74BEFFA61C4413986537C5C58590C4
2016-01-14 17:16:45,590 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:45,590 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:45,590 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:45,590 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:45,590 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:45,590 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:45,590 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 645503360, TotalRequestedScaled: 5.958492553846154E8, TotalJVMHeap: 717225984, TotalAvailable: 480541409, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:45,590 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F4D7C0EE2F92429B98DABF575DD70EAE, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=645503360, allocated=480541409
2016-01-14 17:16:45,590 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 5C74BEFFA61C4413986537C5C58590C4, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:45,590 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:45,590 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F4D7C0EE2F92429B98DABF575DD70EAE being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:45,590 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F4D7C0EE2F92429B98DABF575DD70EAE
2016-01-14 17:16:45,590 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:45,591 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:45,591 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:45,591 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:45,591 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: F4D7C0EE2F92429B98DABF575DD70EAE with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:45,591 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=645503360, postMergeMem=0, RuntimeTotalAvailable=480541409. Updated to: ShuffleMem=480541409, postMergeMem: 0
2016-01-14 17:16:45,591 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=480541409, maxSingleShuffleLimit=120135352, mergeThreshold=432487264, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:45,591 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:45,592 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F4D7C0EE2F92429B98DABF575DD70EAE
2016-01-14 17:16:45,592 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:45,592 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791805333_0001_1_01_000000_0
2016-01-14 17:16:45,594 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791805333_0001_1_00_000000_0_10255, runDuration: 34730, ]
2016-01-14 17:16:45,595 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:45,595 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 381C50FCDEE2484886D230FB43FD3F07
2016-01-14 17:16:45,595 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_SWAP"], id: 5C74BEFFA61C4413986537C5C58590C4
2016-01-14 17:16:45,596 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_SWAP"]
2016-01-14 17:16:45,596 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 381C50FCDEE2484886D230FB43FD3F07, for 1 inputs
2016-01-14 17:16:45,596 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791805333_0001_1_00_000000_0_10255] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791805596, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 17:16:45,596 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : F4D7C0EE2F92429B98DABF575DD70EAE
2016-01-14 17:16:45,596 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:45,597 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [F4D7C0EE2F92429B98DABF575DD70EAE] #43 in 2ms
2016-01-14 17:16:45,597 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [F4D7C0EE2F92429B98DABF575DD70EAE] #43, status:false, isInterrupted:false
2016-01-14 17:16:45,597 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:45,598 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:45,598 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:45,598 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:45,598 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:45,598 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : F4D7C0EE2F92429B98DABF575DD70EAE
2016-01-14 17:16:45,599 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A29043E6066E4C5F9BA9802BF9D2E312, all 1 inputs ready in: 00:00:00.003
2016-01-14 17:16:45,599 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:45,599 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@631b55ce; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@179e7a37
2016-01-14 17:16:45,601 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:45,602 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:45,602 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791805333_0001_1_01_000000_0
2016-01-14 17:16:45,602 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791805333_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:45,603 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791805333_0001_1_01_000000_0
2016-01-14 17:16:45,603 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A29043E6066E4C5F9BA9802BF9D2E312, mem on close (mb), free: 443, total: 684, max: 684
2016-01-14 17:16:45,603 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: F4D7C0EE2F92429B98DABF575DD70EAE
2016-01-14 17:16:45,603 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:45,603 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:45,604 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:45,604 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:45,604 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:45 Completed running task attempt: attempt_1452791805333_0001_1_01_000000_0
2016-01-14 17:16:45,605 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791805333_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:45,605 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791805333_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A29043E6066E4C5F9BA9802BF9D2E312, taskAttemptId=attempt_1452791805333_0001_1_01_000000_0, startTime=1452791805573, finishTime=1452791805605, timeTaken=32, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=717225984, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791805599, Process_Duration=4, Process_End_Time=1452791805603, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:45,605 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791805333_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:45,605 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:45,605 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:45,605 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791805333_0001_1_01_000000_0
2016-01-14 17:16:45,607 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791805333_0001_1][Event:TASK_FINISHED]: vertexName=A29043E6066E4C5F9BA9802BF9D2E312, taskId=task_1452791805333_0001_1_01_000000, startTime=1452791805573, finishTime=1452791805606, timeTaken=33, status=SUCCEEDED, successfulAttemptID=attempt_1452791805333_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=717225984, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791805599, Process_Duration=4, Process_End_Time=1452791805603, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:45,607 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791805333_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:45,607 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=717225984, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791805599, Process_Duration=4, Process_End_Time=1452791805603, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 17:16:45,607 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:45,607 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791805333_0001_1_01 [A29043E6066E4C5F9BA9802BF9D2E312] : 1
2016-01-14 17:16:45,607 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:45,607 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:45,607 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791805333_0001_1_01 [A29043E6066E4C5F9BA9802BF9D2E312]
2016-01-14 17:16:45,607 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791805333_0001_00_000002
2016-01-14 17:16:45,608 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:45,608 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791805333_0001_1][Event:VERTEX_FINISHED]: vertexName=A29043E6066E4C5F9BA9802BF9D2E312, vertexId=vertex_1452791805333_0001_1_01, initRequestedTime=1452791805492, initedTime=1452791805500, startRequestedTime=1452791805507, startedTime=1452791805507, finishTime=1452791805607, timeTaken=100, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=717225984, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791805599, Process_Duration=4, Process_End_Time=1452791805603, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452791805573, firstTasksToStart=[ task_1452791805333_0001_1_01_000000 ], lastTaskFinishTime=1452791805605, lastTasksToFinish=[ task_1452791805333_0001_1_01_000000 ], minTaskDuration=32, maxTaskDuration=32, avgTaskDuration=32.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791805333_0001_1_01_000000 ], longestDurationTasks=[ task_1452791805333_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:45,608 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791805333_0001_1_01 [A29043E6066E4C5F9BA9802BF9D2E312] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:45,608 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791805333_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:45,608 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791805333_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791805333_0001_00_000002, stoppedTime=1452791805608, exitStatus=0
2016-01-14 17:16:45,609 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791805333_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:45,608 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:45,609 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791805333_0001_1_01 [A29043E6066E4C5F9BA9802BF9D2E312] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:45,609 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791805333_0001_1
2016-01-14 17:16:45,609 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791805333_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452791805333_0001_1
2016-01-14 17:16:45,609 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 5C74BEFFA61C4413986537C5C58590C4 for vertex: vertex_1452791805333_0001_1_01 [A29043E6066E4C5F9BA9802BF9D2E312]
2016-01-14 17:16:45,615 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452791805333_0001_1_00 [F4D7C0EE2F92429B98DABF575DD70EAE]
2016-01-14 17:16:45,617 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791805333_0001_1, queueSize=0
2016-01-14 17:16:45,617 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:45,618 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791805333_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 17:16:45,618 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791805333_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:45,618 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791805333_0001_00_000002, Exiting
2016-01-14 17:16:45,618 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791805333_0001_00_000002
2016-01-14 17:16:45,618 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791805333_0001_00_000002
2016-01-14 17:16:45,618 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791805333_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791805333_0001_1, startTime=1452791805491, finishTime=1452791805615, timeTaken=124, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=465, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1434451968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905583611151, Process_Duration=7, Process_End_Time=2905583611158, Read_Duration=2, Tuples_Read=40, Tuples_Written=40, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 17:16:45,618 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791805333_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:45,619 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791805333_0001_00_000002 completed successfully
2016-01-14 17:16:45,619 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791805333_0001_1 finished with state: SUCCEEDED
2016-01-14 17:16:45,619 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791805333_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:45,619 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452791805333_0001_00_000002
2016-01-14 17:16:45,619 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791805333_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:16:45 Completed Dag: dag_1452791805333_0001_1
2016-01-14 17:16:45,619 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791805333_0001_1, dagState=SUCCEEDED
2016-01-14 17:16:45,619 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791805333_0001_1_post
2016-01-14 17:16:45,619 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:45,620 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:45,624 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...platform/ALL_'field'_SWAP, applicationId=application_1452791805333_0001
2016-01-14 17:16:45,625 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:45,626 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:45,626 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:45,626 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:45,626 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:45,626 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:45,626 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:45,627 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:45,627 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:45,627 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:45,628 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:45,628 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:45,628 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:45,632 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/94693635D0844BACA8B4A6A360F6443A/AAA35E222C7F4D85A285413D94B2BCB9/.tez/application_1452791805333_0001
2016-01-14 17:16:45,632 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.316
2016-01-14 17:16:45,703 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_'field'_SWAP
2016-01-14 17:16:45,723 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:45,724 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:45,731 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 17:16:45,731 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 17:16:45,752 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.007
2016-01-14 17:16:45,756 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-01-14 17:16:45,757 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:45,757 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:45,757 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:45,769 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:45,773 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_-1
2016-01-14 17:16:45,812 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:45,812 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:45,813 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_-1"]
2016-01-14 17:16:45,813 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:45,813 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:45,813 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:45,813 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...lyplatform/ALL_UNKNOWN_-1
2016-01-14 17:16:45,813 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:45,824 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:45,828 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:45,828 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:45,828 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:45,837 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A543B16C5BAD45EBA70593B827DDA1DF/B1680A86C1FE4A748F8D53EA7CF4E642/.tez/application_1452791805828_0001 doesn't exist and is created
2016-01-14 17:16:45,852 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:45,852 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:45,853 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A543B16C5BAD45EBA70593B827DDA1DF/B1680A86C1FE4A748F8D53EA7CF4E642/.tez/application_1452791805828_0001
2016-01-14 17:16:45,855 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791805828_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:45,864 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:45,864 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:45,865 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:45,865 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:45,866 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A543B16C5BAD45EBA70593B827DDA1DF/B1680A86C1FE4A748F8D53EA7CF4E642/.tez/application_1452791805828_0001/localmode-log-dir
2016-01-14 17:16:45,866 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A543B16C5BAD45EBA70593B827DDA1DF/B1680A86C1FE4A748F8D53EA7CF4E642/.tez/application_1452791805828_0001/localmode-log-dir/history.txt.appattempt_1452791805828_0001_000000, maxErrors=10
2016-01-14 17:16:45,866 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:45,866 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791805828_0001_000000, appSubmitTime=1452791805854, launchTime=1452791805855
2016-01-14 17:16:45,870 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:45,870 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:45,870 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:33775
2016-01-14 17:16:45,871 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:45,872 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:45,874 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791805828_0001_000000, startTime=1452791805874
2016-01-14 17:16:45,874 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:45,874 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:45,875 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:45,952 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:45,952 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:45,952 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...lyplatform/ALL_UNKNOWN_-1, applicationId=application_1452791805828_0001, dagName=[A543B16C5BAD45EBA70593B827DDA1DF/B1680A86C1FE4A748F8D53EA7CF4E642] every/(1/1) ...lyplatform/ALL_UNKNOWN_-1
2016-01-14 17:16:45,966 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [A543B16C5BAD45EBA70593B827DDA1DF/B1680A86C1FE4A748F8D53EA7CF4E642] every/(1/1) ...lyplatform/ALL_UNKNOWN_-1
2016-01-14 17:16:45,967 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791805828_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A543B16C5BAD45EBA70593B827DDA1DF/B1680A86C1FE4A748F8D53EA7CF4E642/.tez/application_1452791805828_0001/localmode-log-dir/dag_1452791805828_0001_1.dot
2016-01-14 17:16:45,968 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A543B16C5BAD45EBA70593B827DDA1DF/B1680A86C1FE4A748F8D53EA7CF4E642/.tez/application_1452791805828_0001/localmode-log-dir/dag_1452791805828_0001_1-tez-dag.pb.txt
2016-01-14 17:16:45,973 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791805828_0001_1
2016-01-14 17:16:45,973 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:45,973 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [A543B16C5BAD45EBA70593B827DDA1DF/B1680A86C1FE4A748F8D53EA7CF4E642] every/(1/1) ...lyplatform/ALL_UNKNOWN_-1
2016-01-14 17:16:45 Running Dag: dag_1452791805828_0001_1
2016-01-14 17:16:45,984 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791805828_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791805828_0001_1, submitTime=1452791805966
2016-01-14 17:16:45,984 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:45,985 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex D9EEA6E4044E443CA6C12D9D51282C88
2016-01-14 17:16:45,985 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:45,985 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791805828_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791805828_0001_1, initTime=1452791805984
2016-01-14 17:16:45,985 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791805828_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:45,985 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:45,986 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791805828_0001_1][Event:DAG_STARTED]: dagID=dag_1452791805828_0001_1, startTime=1452791805986
2016-01-14 17:16:45,986 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:45,986 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:45,986 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...lyplatform/ALL_UNKNOWN_-1, applicationId=application_1452791805828_0001, dagName=[A543B16C5BAD45EBA70593B827DDA1DF/B1680A86C1FE4A748F8D53EA7CF4E642] every/(1/1) ...lyplatform/ALL_UNKNOWN_-1
2016-01-14 17:16:45,986 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791805828_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:45,988 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: FCDE3E3D29FB4F71A7F39D8A4E30E4DC : {5950C7CB8F1D49D49680C0B1B73C5636={InputName=5950C7CB8F1D49D49680C0B1B73C5636}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:45,988 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 5950C7CB8F1D49D49680C0B1B73C5636, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:45,988 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791805828_0001_1_00 [FCDE3E3D29FB4F71A7F39D8A4E30E4DC]
2016-01-14 17:16:45,988 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791805828_0001_1_00 [FCDE3E3D29FB4F71A7F39D8A4E30E4DC]
2016-01-14 17:16:45,988 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791805828_0001_1_00 [FCDE3E3D29FB4F71A7F39D8A4E30E4DC]
2016-01-14 17:16:45,989 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791805828_0001_1_00 [FCDE3E3D29FB4F71A7F39D8A4E30E4DC]. Starting root input initializers: 1
2016-01-14 17:16:45,989 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791805828_0001_1_00 [FCDE3E3D29FB4F71A7F39D8A4E30E4DC] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:45,989 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791805828_0001_1_01 [D9EEA6E4044E443CA6C12D9D51282C88]
2016-01-14 17:16:45,991 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 5950C7CB8F1D49D49680C0B1B73C5636 on vertex vertex_1452791805828_0001_1_00 [FCDE3E3D29FB4F71A7F39D8A4E30E4DC]
2016-01-14 17:16:45,992 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:45,992 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791805828_0001_1_01 [D9EEA6E4044E443CA6C12D9D51282C88]
2016-01-14 17:16:45,992 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791805828_0001_1_01 [D9EEA6E4044E443CA6C12D9D51282C88]
2016-01-14 17:16:45,992 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791805828_0001_1_01 [D9EEA6E4044E443CA6C12D9D51282C88]
2016-01-14 17:16:45,993 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=6E8A539167EF46DF88313BB3CF48CA29, vertexId=vertex_1452791805828_0001_1_01 [D9EEA6E4044E443CA6C12D9D51282C88], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:45,993 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=6E8A539167EF46DF88313BB3CF48CA29, vertexId=vertex_1452791805828_0001_1_01 [D9EEA6E4044E443CA6C12D9D51282C88]
2016-01-14 17:16:45,994 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:45,994 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 5950C7CB8F1D49D49680C0B1B73C5636 asking for 1 tasks. Headroom: 684 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:45,994 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:45,995 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=6E8A539167EF46DF88313BB3CF48CA29, vertexName=D9EEA6E4044E443CA6C12D9D51282C88, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:45,995 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=6E8A539167EF46DF88313BB3CF48CA29, vertexId=vertex_1452791805828_0001_1_01 [D9EEA6E4044E443CA6C12D9D51282C88]
2016-01-14 17:16:45,995 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:45,996 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:45,996 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791805828_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D9EEA6E4044E443CA6C12D9D51282C88, vertexId=vertex_1452791805828_0001_1_01, initRequestedTime=1452791805989, initedTime=1452791805996, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:45,997 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791805828_0001_1_01 [D9EEA6E4044E443CA6C12D9D51282C88] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:45,997 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:45,998 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 5950C7CB8F1D49D49680C0B1B73C5636 on vertex vertex_1452791805828_0001_1_00 [FCDE3E3D29FB4F71A7F39D8A4E30E4DC]
2016-01-14 17:16:45,998 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791805828_0001_1_00 [FCDE3E3D29FB4F71A7F39D8A4E30E4DC] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:45,998 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {5950C7CB8F1D49D49680C0B1B73C5636=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:45,998 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791805828_0001_1_00 [FCDE3E3D29FB4F71A7F39D8A4E30E4DC] parallelism set to 1
2016-01-14 17:16:45,999 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791805828_0001_1_00 [FCDE3E3D29FB4F71A7F39D8A4E30E4DC] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:45,999 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791805828_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791805828_0001_1_00, eventCount=1
2016-01-14 17:16:45,999 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791805828_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FCDE3E3D29FB4F71A7F39D8A4E30E4DC, vertexId=vertex_1452791805828_0001_1_00, initRequestedTime=1452791805988, initedTime=1452791805999, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:46,000 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791805828_0001_1_00 [FCDE3E3D29FB4F71A7F39D8A4E30E4DC] with distanceFromRoot: 0
2016-01-14 17:16:46,000 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791805828_0001_1_00 [FCDE3E3D29FB4F71A7F39D8A4E30E4DC] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:46,000 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in FCDE3E3D29FB4F71A7F39D8A4E30E4DC
2016-01-14 17:16:46,000 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791805828_0001_1_00 [FCDE3E3D29FB4F71A7F39D8A4E30E4DC]
2016-01-14 17:16:45,999 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:46,000 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791805828_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791805828_0001_1_00, startRequestedTime=1452791806000, startedTime=1452791806000
2016-01-14 17:16:46,001 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:46,001 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791805828_0001_1_00 [FCDE3E3D29FB4F71A7F39D8A4E30E4DC] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:46,001 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:46,002 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791805828_0001_1][Event:TASK_STARTED]: vertexName=FCDE3E3D29FB4F71A7F39D8A4E30E4DC, taskId=task_1452791805828_0001_1_00_000000, scheduledTime=1452791806001, launchTime=1452791806001
2016-01-14 17:16:46,002 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791805828_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:46,002 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791805828_0001_1_00 for vertex: vertex_1452791805828_0001_1_01 [D9EEA6E4044E443CA6C12D9D51282C88] numStartedSources: 1 numSources: 1
2016-01-14 17:16:46,002 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791805828_0001_1_01 [D9EEA6E4044E443CA6C12D9D51282C88] with distanceFromRoot: 1
2016-01-14 17:16:46,002 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791805828_0001_1_00 [FCDE3E3D29FB4F71A7F39D8A4E30E4DC]
2016-01-14 17:16:46,003 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791805828_0001, with dag id: dag_1452791805828_0001_1
2016-01-14 17:16:46,003 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: FCDE3E3D29FB4F71A7F39D8A4E30E4DC in vertex: D9EEA6E4044E443CA6C12D9D51282C88
2016-01-14 17:16:46,003 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: D9EEA6E4044E443CA6C12D9D51282C88 with 1 source tasks and 1 pending tasks
2016-01-14 17:16:46,003 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:46,003 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791805828_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791805828_0001_1_01, startRequestedTime=1452791806003, startedTime=1452791806003
2016-01-14 17:16:46,003 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791805828_0001_1_01 [D9EEA6E4044E443CA6C12D9D51282C88] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:46,004 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [A543B16C5BAD45EBA70593B827DDA1DF/B1680A86C1FE4A748F8D53EA7CF4E642] every/(1/1) ...lyplatform/ALL_UNKNOWN_-1, VertexName: FCDE3E3D29FB4F71A7F39D8A4E30E4DC, VertexParallelism: 1, TaskAttemptID:attempt_1452791805828_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=5950C7CB8F1D49D49680C0B1B73C5636, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D9EEA6E4044E443CA6C12D9D51282C88, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:46,004 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791805828_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:46,004 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:46,004 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:46,005 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:46,005 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:46,005 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791805828_0001_00_000001
2016-01-14 17:16:46,006 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791805828_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:46,007 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791805828_0001_00_000001
2016-01-14 17:16:46,007 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:46,008 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791805828_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:46,008 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:46,008 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791805828_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791805828_0001_00_000001, launchTime=1452791806008
2016-01-14 17:16:46,008 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791805828_0001_00_000001
2016-01-14 17:16:46,009 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791805828_0001_1_00_000000_0] to container: [container_1452791805828_0001_00_000001]
2016-01-14 17:16:46,009 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791805828_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:46,009 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791805828_0001_00_000001 given task: attempt_1452791805828_0001_1_00_000000_0
2016-01-14 17:16:46,009 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791805828_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791805828_0001_1_00_000000_0
2016-01-14 17:16:46,010 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791805828_0001_1_00_000000_0] started. Is using containerId: [container_1452791805828_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:46,010 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:46,010 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791805828_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FCDE3E3D29FB4F71A7F39D8A4E30E4DC, taskAttemptId=attempt_1452791805828_0001_1_00_000000_0, startTime=1452791806010, containerId=container_1452791805828_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791805828_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:46,010 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791805828_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:46,010 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791805828_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:46 Starting to run new task attempt: attempt_1452791805828_0001_1_00_000000_0
2016-01-14 17:16:46,011 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791805828_0001_1_00_000000_0
2016-01-14 17:16:46,011 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:46,011 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:46,011 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:46,011 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:46,011 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:46,012 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:46,012 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [A543B16C5BAD45EBA70593B827DDA1DF/B1680A86C1FE4A748F8D53EA7CF4E642] every/(1/1) ...lyplatform/ALL_UNKNOWN_-1, VertexName: FCDE3E3D29FB4F71A7F39D8A4E30E4DC, VertexParallelism: 1, TaskAttemptID:attempt_1452791805828_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=5950C7CB8F1D49D49680C0B1B73C5636, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D9EEA6E4044E443CA6C12D9D51282C88, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:46,012 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=717225984
2016-01-14 17:16:46,012 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791805828_0001_1_00_000000_0
2016-01-14 17:16:46,012 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:46,012 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:46,013 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791805828_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:46,014 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=5950C7CB8F1D49D49680C0B1B73C5636, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:46,014 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:46,014 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 5950C7CB8F1D49D49680C0B1B73C5636
2016-01-14 17:16:46,014 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 5950C7CB8F1D49D49680C0B1B73C5636
2016-01-14 17:16:46,016 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:46,016 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 5950C7CB8F1D49D49680C0B1B73C5636
2016-01-14 17:16:46,016 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 5950C7CB8F1D49D49680C0B1B73C5636
2016-01-14 17:16:46,016 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:46,019 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:46,019 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=D9EEA6E4044E443CA6C12D9D51282C88, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:46,019 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:46,019 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: D9EEA6E4044E443CA6C12D9D51282C88
2016-01-14 17:16:46,021 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:46,021 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: D9EEA6E4044E443CA6C12D9D51282C88
2016-01-14 17:16:46,021 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: D9EEA6E4044E443CA6C12D9D51282C88
2016-01-14 17:16:46,022 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FCDE3E3D29FB4F71A7F39D8A4E30E4DC, ordinal: 0
2016-01-14 17:16:46,022 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FCDE3E3D29FB4F71A7F39D8A4E30E4DC, mem on start (mb), free: 539, total: 684, max: 684
2016-01-14 17:16:46,022 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:46,022 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:46,022 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:46,023 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:46,023 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:46,023 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:46,023 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:46,023 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:46,023 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:46,024 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 717225984, TotalAvailable: 480541409, TotalRequested/TotalJVMHeap:0.15
2016-01-14 17:16:46,024 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 5950C7CB8F1D49D49680C0B1B73C5636, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:46,024 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, D9EEA6E4044E443CA6C12D9D51282C88, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:46,024 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:46,024 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 5950C7CB8F1D49D49680C0B1B73C5636
2016-01-14 17:16:46,024 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:46,024 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 5950C7CB8F1D49D49680C0B1B73C5636
2016-01-14 17:16:46,024 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 5950C7CB8F1D49D49680C0B1B73C5636 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:46,025 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:46,025 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:46,025 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:46,025 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791805828_0001_1_00_000000_0
2016-01-14 17:16:46,025 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:46,026 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@70b0c569
2016-01-14 17:16:46,027 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@70b0c569
2016-01-14 17:16:46,027 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:46,027 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:46,028 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:46,028 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 5950C7CB8F1D49D49680C0B1B73C5636
2016-01-14 17:16:46,028 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: C5D10A9B94484BEF9451783864587D93
2016-01-14 17:16:46,028 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] C5D10A9B94484BEF9451783864587D93
2016-01-14 17:16:46,029 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@7900fc82; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@2fc1f8de; comparator=cascading.tuple.hadoop.util.TupleComparator@67b9cf50; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:46,029 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:46,029 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:46,029 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:46,042 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:46,043 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:46,043 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:46,043 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:46,043 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:46,044 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:46,044 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:46,044 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FCDE3E3D29FB4F71A7F39D8A4E30E4DC, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:46,048 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791805828_0001_1_00_000000_0
2016-01-14 17:16:46,048 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FCDE3E3D29FB4F71A7F39D8A4E30E4DC, mem on close (mb), free: 437, total: 684, max: 684
2016-01-14 17:16:46,048 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:46,048 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:46,048 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:46,049 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:46,118 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:46 Completed running task attempt: attempt_1452791805828_0001_1_00_000000_0
2016-01-14 17:16:46,121 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791805828_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:46,121 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791805828_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791805828_0001_1_00, eventCount=1
2016-01-14 17:16:46,121 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=67, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=717750272, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791806044, Process_Duration=4, Process_End_Time=1452791806048, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=2][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:46,122 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:46,122 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:46,122 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791805828_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FCDE3E3D29FB4F71A7F39D8A4E30E4DC, taskAttemptId=attempt_1452791805828_0001_1_00_000000_0, startTime=1452791806010, finishTime=1452791806122, timeTaken=112, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=67, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=717750272, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791806044, Process_Duration=4, Process_End_Time=1452791806048, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:46,122 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:46,122 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791805828_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:46,123 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791805828_0001_00_000001
2016-01-14 17:16:46,123 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791805828_0001_1_00_000000_0
2016-01-14 17:16:46,123 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791805828_0001_1][Event:TASK_FINISHED]: vertexName=FCDE3E3D29FB4F71A7F39D8A4E30E4DC, taskId=task_1452791805828_0001_1_00_000000, startTime=1452791806010, finishTime=1452791806123, timeTaken=113, status=SUCCEEDED, successfulAttemptID=attempt_1452791805828_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=67, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=717750272, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791806044, Process_Duration=4, Process_End_Time=1452791806048, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:46,123 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791805828_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:46,123 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791805828_0001_1_00 [FCDE3E3D29FB4F71A7F39D8A4E30E4DC] : 1
2016-01-14 17:16:46,123 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:46,123 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:46,123 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791805828_0001_1_00 [FCDE3E3D29FB4F71A7F39D8A4E30E4DC]
2016-01-14 17:16:46,124 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:46,124 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791805828_0001_1][Event:VERTEX_FINISHED]: vertexName=FCDE3E3D29FB4F71A7F39D8A4E30E4DC, vertexId=vertex_1452791805828_0001_1_00, initRequestedTime=1452791805988, initedTime=1452791805999, startRequestedTime=1452791806000, startedTime=1452791806000, finishTime=1452791806124, timeTaken=124, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=67, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=717750272, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791806044, Process_Duration=4, Process_End_Time=1452791806048, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791806010, firstTasksToStart=[ task_1452791805828_0001_1_00_000000 ], lastTaskFinishTime=1452791806122, lastTasksToFinish=[ task_1452791805828_0001_1_00_000000 ], minTaskDuration=112, maxTaskDuration=112, avgTaskDuration=112.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791805828_0001_1_00_000000 ], longestDurationTasks=[ task_1452791805828_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:46,124 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791805828_0001_1_00 [FCDE3E3D29FB4F71A7F39D8A4E30E4DC] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:46,125 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791805828_0001_1_01 [D9EEA6E4044E443CA6C12D9D51282C88] attempt: attempt_1452791805828_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:46,125 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: D9EEA6E4044E443CA6C12D9D51282C88
2016-01-14 17:16:46,125 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791805828_0001_1_01 [D9EEA6E4044E443CA6C12D9D51282C88]
2016-01-14 17:16:46,125 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791805828_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:46,125 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791805828_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791805828_0001_00_000001, stoppedTime=1452791806125, exitStatus=0
2016-01-14 17:16:46,125 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791805828_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:46,125 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791805828_0001_1_00 [FCDE3E3D29FB4F71A7F39D8A4E30E4DC] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:46,126 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791805828_0001_1][Event:TASK_STARTED]: vertexName=D9EEA6E4044E443CA6C12D9D51282C88, taskId=task_1452791805828_0001_1_01_000000, scheduledTime=1452791806126, launchTime=1452791806126
2016-01-14 17:16:46,126 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791805828_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:46,125 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:46,126 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:46,126 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791805828_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:46,126 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791805828_0001_1_01 [D9EEA6E4044E443CA6C12D9D51282C88]
2016-01-14 17:16:46,126 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791805828_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:46,126 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [A543B16C5BAD45EBA70593B827DDA1DF/B1680A86C1FE4A748F8D53EA7CF4E642] every/(1/1) ...lyplatform/ALL_UNKNOWN_-1, VertexName: D9EEA6E4044E443CA6C12D9D51282C88, VertexParallelism: 1, TaskAttemptID:attempt_1452791805828_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FCDE3E3D29FB4F71A7F39D8A4E30E4DC, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=6E8A539167EF46DF88313BB3CF48CA29, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:46,126 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791805828_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:46,127 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:46,127 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791805828_0001_00_000002
2016-01-14 17:16:46,127 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791805828_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:46,129 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791805828_0001_00_000002
2016-01-14 17:16:46,129 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:46,129 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791805828_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:46,129 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791805828_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791805828_0001_00_000002, launchTime=1452791806129
2016-01-14 17:16:46,129 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:46,130 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791805828_0001_00_000002
2016-01-14 17:16:46,130 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791805828_0001_1_01_000000_0] to container: [container_1452791805828_0001_00_000002]
2016-01-14 17:16:46,130 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791805828_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:46,130 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791805828_0001_00_000002 given task: attempt_1452791805828_0001_1_01_000000_0
2016-01-14 17:16:46,131 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791805828_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791805828_0001_1_01_000000_0
2016-01-14 17:16:46,131 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791805828_0001_1_01_000000_0] started. Is using containerId: [container_1452791805828_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:46,131 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:46 Starting to run new task attempt: attempt_1452791805828_0001_1_01_000000_0
2016-01-14 17:16:46,131 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791805828_0001_1_01_000000_0
2016-01-14 17:16:46,132 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:46,132 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:46,132 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:46,132 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:46,132 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:46,131 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791805828_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D9EEA6E4044E443CA6C12D9D51282C88, taskAttemptId=attempt_1452791805828_0001_1_01_000000_0, startTime=1452791806131, containerId=container_1452791805828_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791805828_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:46,132 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791805828_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:46,132 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:46,132 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791805828_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:46,132 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [A543B16C5BAD45EBA70593B827DDA1DF/B1680A86C1FE4A748F8D53EA7CF4E642] every/(1/1) ...lyplatform/ALL_UNKNOWN_-1, VertexName: D9EEA6E4044E443CA6C12D9D51282C88, VertexParallelism: 1, TaskAttemptID:attempt_1452791805828_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FCDE3E3D29FB4F71A7F39D8A4E30E4DC, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=6E8A539167EF46DF88313BB3CF48CA29, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:46,132 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=717750272
2016-01-14 17:16:46,133 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791805828_0001_1_01_000000_0
2016-01-14 17:16:46,133 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:46,133 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791805828_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:46,133 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791805828_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:46,133 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:46,133 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791805828_0001_00_000001, Exiting
2016-01-14 17:16:46,133 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791805828_0001_00_000001
2016-01-14 17:16:46,134 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791805828_0001_00_000001
2016-01-14 17:16:46,133 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791805828_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:46,134 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:46,136 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=6E8A539167EF46DF88313BB3CF48CA29, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:46,136 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:46,136 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 6E8A539167EF46DF88313BB3CF48CA29
2016-01-14 17:16:46,136 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:46,136 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 6E8A539167EF46DF88313BB3CF48CA29
2016-01-14 17:16:46,136 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:46,139 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D9EEA6E4044E443CA6C12D9D51282C88, ordinal: 1
2016-01-14 17:16:46,139 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D9EEA6E4044E443CA6C12D9D51282C88, mem on start (mb), free: 426, total: 684, max: 684
2016-01-14 17:16:46,139 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:46,139 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:46,140 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=FCDE3E3D29FB4F71A7F39D8A4E30E4DC, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:46,140 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:46,140 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: FCDE3E3D29FB4F71A7F39D8A4E30E4DC
2016-01-14 17:16:46,141 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 645975232, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:46,141 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:46,141 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 645975232, type: INPUT, componentVertexName: FCDE3E3D29FB4F71A7F39D8A4E30E4DC
2016-01-14 17:16:46,141 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: FCDE3E3D29FB4F71A7F39D8A4E30E4DC
2016-01-14 17:16:46,142 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:46,142 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791805828_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:46,142 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791805828_0001_00_000001 completed successfully
2016-01-14 17:16:46,142 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791805828_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:46,147 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:46,147 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 6E8A539167EF46DF88313BB3CF48CA29
2016-01-14 17:16:46,147 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:46,147 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:46,148 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:46,148 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:46,148 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:46,148 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:46,148 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 645975232, TotalRequestedScaled: 5.962848295384616E8, TotalJVMHeap: 717750272, TotalAvailable: 480892682, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:46,148 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 6E8A539167EF46DF88313BB3CF48CA29, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:46,148 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, FCDE3E3D29FB4F71A7F39D8A4E30E4DC, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=645975232, allocated=480892682
2016-01-14 17:16:46,148 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:46,148 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: FCDE3E3D29FB4F71A7F39D8A4E30E4DC being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:46,148 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:46,148 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:46,149 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: FCDE3E3D29FB4F71A7F39D8A4E30E4DC
2016-01-14 17:16:46,149 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:46,149 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:46,150 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: FCDE3E3D29FB4F71A7F39D8A4E30E4DC with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:46,150 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=645975232, postMergeMem=0, RuntimeTotalAvailable=480892682. Updated to: ShuffleMem=480892682, postMergeMem: 0
2016-01-14 17:16:46,150 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=480892682, maxSingleShuffleLimit=120223168, mergeThreshold=432803392, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:46,150 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:46,151 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: FCDE3E3D29FB4F71A7F39D8A4E30E4DC
2016-01-14 17:16:46,151 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:46,151 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791805828_0001_1_01_000000_0
2016-01-14 17:16:46,152 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791805828_0001_1_00_000000_0_10261, runDuration: 100562, ]
2016-01-14 17:16:46,153 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791805828_0001_1_00_000000_0_10261] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791806153, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 17:16:46,153 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : FCDE3E3D29FB4F71A7F39D8A4E30E4DC
2016-01-14 17:16:46,153 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:46,153 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [FCDE3E3D29FB4F71A7F39D8A4E30E4DC] #44 in 1ms
2016-01-14 17:16:46,153 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [FCDE3E3D29FB4F71A7F39D8A4E30E4DC] #44, status:false, isInterrupted:false
2016-01-14 17:16:46,154 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:46,154 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:46,154 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: C5D10A9B94484BEF9451783864587D93
2016-01-14 17:16:46,154 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:46,154 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:46,154 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:46,154 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_-1"], id: 6E8A539167EF46DF88313BB3CF48CA29
2016-01-14 17:16:46,155 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_-1"]
2016-01-14 17:16:46,155 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:46,155 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : FCDE3E3D29FB4F71A7F39D8A4E30E4DC
2016-01-14 17:16:46,155 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:46,156 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] C5D10A9B94484BEF9451783864587D93, for 1 inputs
2016-01-14 17:16:46,157 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D9EEA6E4044E443CA6C12D9D51282C88, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:46,157 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@4eeb7f13; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@46e3e0df
2016-01-14 17:16:46,160 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:46,161 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:46,161 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791805828_0001_1_01_000000_0
2016-01-14 17:16:46,161 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791805828_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:46,162 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791805828_0001_1_01_000000_0
2016-01-14 17:16:46,162 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D9EEA6E4044E443CA6C12D9D51282C88, mem on close (mb), free: 422, total: 684, max: 684
2016-01-14 17:16:46,162 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: FCDE3E3D29FB4F71A7F39D8A4E30E4DC
2016-01-14 17:16:46,162 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:46,163 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:46,163 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:46,163 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:46,163 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:46 Completed running task attempt: attempt_1452791805828_0001_1_01_000000_0
2016-01-14 17:16:46,164 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791805828_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:46,165 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=717750272, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791806157, Process_Duration=5, Process_End_Time=1452791806162, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 17:16:46,165 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:46,165 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791805828_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D9EEA6E4044E443CA6C12D9D51282C88, taskAttemptId=attempt_1452791805828_0001_1_01_000000_0, startTime=1452791806131, finishTime=1452791806165, timeTaken=34, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=717750272, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791806157, Process_Duration=5, Process_End_Time=1452791806162, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:46,165 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791805828_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:46,165 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791805828_0001_1_01_000000_0
2016-01-14 17:16:46,166 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791805828_0001_1][Event:TASK_FINISHED]: vertexName=D9EEA6E4044E443CA6C12D9D51282C88, taskId=task_1452791805828_0001_1_01_000000, startTime=1452791806131, finishTime=1452791806165, timeTaken=34, status=SUCCEEDED, successfulAttemptID=attempt_1452791805828_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=717750272, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791806157, Process_Duration=5, Process_End_Time=1452791806162, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:46,166 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791805828_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:46,166 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791805828_0001_1_01 [D9EEA6E4044E443CA6C12D9D51282C88] : 1
2016-01-14 17:16:46,166 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791805828_0001_1_01 [D9EEA6E4044E443CA6C12D9D51282C88]
2016-01-14 17:16:46,166 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:46,166 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:46,166 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:46,166 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:46,166 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:46,167 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791805828_0001_1][Event:VERTEX_FINISHED]: vertexName=D9EEA6E4044E443CA6C12D9D51282C88, vertexId=vertex_1452791805828_0001_1_01, initRequestedTime=1452791805989, initedTime=1452791805996, startRequestedTime=1452791806003, startedTime=1452791806003, finishTime=1452791806166, timeTaken=163, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=717750272, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791806157, Process_Duration=5, Process_End_Time=1452791806162, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452791806131, firstTasksToStart=[ task_1452791805828_0001_1_01_000000 ], lastTaskFinishTime=1452791806165, lastTasksToFinish=[ task_1452791805828_0001_1_01_000000 ], minTaskDuration=34, maxTaskDuration=34, avgTaskDuration=34.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791805828_0001_1_01_000000 ], longestDurationTasks=[ task_1452791805828_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:46,166 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791805828_0001_00_000002
2016-01-14 17:16:46,167 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791805828_0001_1_01 [D9EEA6E4044E443CA6C12D9D51282C88] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:46,167 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791805828_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:46,167 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791805828_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791805828_0001_00_000002, stoppedTime=1452791806167, exitStatus=0
2016-01-14 17:16:46,167 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791805828_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:46,167 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791805828_0001_1_01 [D9EEA6E4044E443CA6C12D9D51282C88] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:46,167 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791805828_0001_1
2016-01-14 17:16:46,167 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791805828_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452791805828_0001_1
2016-01-14 17:16:46,167 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 6E8A539167EF46DF88313BB3CF48CA29 for vertex: vertex_1452791805828_0001_1_01 [D9EEA6E4044E443CA6C12D9D51282C88]
2016-01-14 17:16:46,168 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:46,174 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452791805828_0001_1_00 [FCDE3E3D29FB4F71A7F39D8A4E30E4DC]
2016-01-14 17:16:46,176 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791805828_0001_1, queueSize=0
2016-01-14 17:16:46,176 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:46,177 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791805828_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791805828_0001_1, startTime=1452791805986, finishTime=1452791806175, timeTaken=189, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=465, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=67, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1435500544, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905583612201, Process_Duration=9, Process_End_Time=2905583612210, Read_Duration=4, Tuples_Read=40, Tuples_Written=40, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 17:16:46,177 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791805828_0001_1 finished with state: SUCCEEDED
2016-01-14 17:16:46,177 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791805828_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:46,177 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791805828_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 17:16:46,177 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791805828_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:46,178 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791805828_0001_00_000002, Exiting
2016-01-14 17:16:46,178 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791805828_0001_00_000002
2016-01-14 17:16:46,178 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791805828_0001_00_000002
2016-01-14 17:16:46,178 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791805828_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:46,178 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791805828_0001_00_000002 completed successfully
2016-01-14 17:16:46,178 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452791805828_0001_00_000002
2016-01-14 17:16:46 Completed Dag: dag_1452791805828_0001_1
2016-01-14 17:16:46,182 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791805828_0001_1, dagState=SUCCEEDED
2016-01-14 17:16:46,182 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791805828_0001_1_post
2016-01-14 17:16:46,182 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:46,182 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:46,183 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791805828_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:16:46,185 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 17:16:46,185 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 17:16:46,186 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...lyplatform/ALL_UNKNOWN_-1, applicationId=application_1452791805828_0001
2016-01-14 17:16:46,187 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:46,187 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:46,187 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:46,187 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:46,188 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:46,188 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:46,188 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:46,189 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:46,189 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:46,189 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:46,189 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:46,189 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:46,189 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:46,193 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A543B16C5BAD45EBA70593B827DDA1DF/B1680A86C1FE4A748F8D53EA7CF4E642/.tez/application_1452791805828_0001
2016-01-14 17:16:46,193 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.381
2016-01-14 17:16:46,259 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_UNKNOWN_-1
2016-01-14 17:16:46,278 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:46,278 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:46,301 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.002
2016-01-14 17:16:46,302 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.003
2016-01-14 17:16:46,302 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 17:16:46,302 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 17:16:46,303 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_UNKNOWN_'field'
2016-01-14 17:16:46,321 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:46,321 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:46,349 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.006
2016-01-14 17:16:46,354 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-01-14 17:16:46,354 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:46,354 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:46,354 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:46,367 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:46,371 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_RESULTS
2016-01-14 17:16:46,408 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:46,409 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:46,409 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_RESULTS"]
2016-01-14 17:16:46,409 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:46,409 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:46,409 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:46,409 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...tform/ALL_UNKNOWN_RESULTS
2016-01-14 17:16:46,409 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:46,420 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:46,424 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:46,424 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:46,424 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:46,433 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5CC4023286854947A855F159EBADB99E/E0933C3C2A7D42638CB8D3E3023CCA8E/.tez/application_1452791806424_0001 doesn't exist and is created
2016-01-14 17:16:46,448 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:46,448 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:46,449 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5CC4023286854947A855F159EBADB99E/E0933C3C2A7D42638CB8D3E3023CCA8E/.tez/application_1452791806424_0001
2016-01-14 17:16:46,451 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791806424_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:46,461 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:46,462 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:46,464 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:46,464 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:46,464 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5CC4023286854947A855F159EBADB99E/E0933C3C2A7D42638CB8D3E3023CCA8E/.tez/application_1452791806424_0001/localmode-log-dir
2016-01-14 17:16:46,464 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5CC4023286854947A855F159EBADB99E/E0933C3C2A7D42638CB8D3E3023CCA8E/.tez/application_1452791806424_0001/localmode-log-dir/history.txt.appattempt_1452791806424_0001_000000, maxErrors=10
2016-01-14 17:16:46,464 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:46,464 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791806424_0001_000000, appSubmitTime=1452791806451, launchTime=1452791806451
2016-01-14 17:16:46,469 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:57264
2016-01-14 17:16:46,472 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:46,472 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:46,472 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:46,473 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:46,473 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:46,475 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791806424_0001_000000, startTime=1452791806475
2016-01-14 17:16:46,475 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:46,475 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:46,548 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:46,548 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:46,548 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...tform/ALL_UNKNOWN_RESULTS, applicationId=application_1452791806424_0001, dagName=[5CC4023286854947A855F159EBADB99E/E0933C3C2A7D42638CB8D3E3023CCA8E] every/(1/1) ...tform/ALL_UNKNOWN_RESULTS
2016-01-14 17:16:46,563 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [5CC4023286854947A855F159EBADB99E/E0933C3C2A7D42638CB8D3E3023CCA8E] every/(1/1) ...tform/ALL_UNKNOWN_RESULTS
2016-01-14 17:16:46,564 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791806424_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5CC4023286854947A855F159EBADB99E/E0933C3C2A7D42638CB8D3E3023CCA8E/.tez/application_1452791806424_0001/localmode-log-dir/dag_1452791806424_0001_1.dot
2016-01-14 17:16:46,564 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5CC4023286854947A855F159EBADB99E/E0933C3C2A7D42638CB8D3E3023CCA8E/.tez/application_1452791806424_0001/localmode-log-dir/dag_1452791806424_0001_1-tez-dag.pb.txt
2016-01-14 17:16:46,570 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791806424_0001_1
2016-01-14 17:16:46,570 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:46,570 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [5CC4023286854947A855F159EBADB99E/E0933C3C2A7D42638CB8D3E3023CCA8E] every/(1/1) ...tform/ALL_UNKNOWN_RESULTS
2016-01-14 17:16:46 Running Dag: dag_1452791806424_0001_1
2016-01-14 17:16:46,581 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791806424_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791806424_0001_1, submitTime=1452791806563
2016-01-14 17:16:46,581 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:46,582 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex E501422D87154B5F84C18C62A15C5355
2016-01-14 17:16:46,582 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:46,582 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:46,583 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791806424_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791806424_0001_1, initTime=1452791806581
2016-01-14 17:16:46,583 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791806424_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:46,583 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791806424_0001_1][Event:DAG_STARTED]: dagID=dag_1452791806424_0001_1, startTime=1452791806583
2016-01-14 17:16:46,583 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:46,583 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:46,584 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791806424_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:46,584 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: AF9B90FAA50E444D8556A417EAE7B3A7 : {C0D7A89BDE404087BEAF7E8D55A48092={InputName=C0D7A89BDE404087BEAF7E8D55A48092}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:46,584 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: C0D7A89BDE404087BEAF7E8D55A48092, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:46,584 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791806424_0001_1_00 [AF9B90FAA50E444D8556A417EAE7B3A7]
2016-01-14 17:16:46,584 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791806424_0001_1_00 [AF9B90FAA50E444D8556A417EAE7B3A7]
2016-01-14 17:16:46,584 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791806424_0001_1_00 [AF9B90FAA50E444D8556A417EAE7B3A7]
2016-01-14 17:16:46,584 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...tform/ALL_UNKNOWN_RESULTS, applicationId=application_1452791806424_0001, dagName=[5CC4023286854947A855F159EBADB99E/E0933C3C2A7D42638CB8D3E3023CCA8E] every/(1/1) ...tform/ALL_UNKNOWN_RESULTS
2016-01-14 17:16:46,584 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791806424_0001_1_00 [AF9B90FAA50E444D8556A417EAE7B3A7]. Starting root input initializers: 1
2016-01-14 17:16:46,585 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791806424_0001_1_00 [AF9B90FAA50E444D8556A417EAE7B3A7] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:46,585 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791806424_0001_1_01 [E501422D87154B5F84C18C62A15C5355]
2016-01-14 17:16:46,585 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: C0D7A89BDE404087BEAF7E8D55A48092 on vertex vertex_1452791806424_0001_1_00 [AF9B90FAA50E444D8556A417EAE7B3A7]
2016-01-14 17:16:46,587 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:46,587 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input C0D7A89BDE404087BEAF7E8D55A48092 asking for 1 tasks. Headroom: 684 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:46,587 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:46,588 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:46,588 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:46,588 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791806424_0001_1_01 [E501422D87154B5F84C18C62A15C5355]
2016-01-14 17:16:46,588 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791806424_0001_1_01 [E501422D87154B5F84C18C62A15C5355]
2016-01-14 17:16:46,588 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791806424_0001_1_01 [E501422D87154B5F84C18C62A15C5355]
2016-01-14 17:16:46,588 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=EDCF24996B0141469E6153427F935E75, vertexId=vertex_1452791806424_0001_1_01 [E501422D87154B5F84C18C62A15C5355], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:46,589 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=EDCF24996B0141469E6153427F935E75, vertexId=vertex_1452791806424_0001_1_01 [E501422D87154B5F84C18C62A15C5355]
2016-01-14 17:16:46,590 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=EDCF24996B0141469E6153427F935E75, vertexName=E501422D87154B5F84C18C62A15C5355, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:46,590 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=EDCF24996B0141469E6153427F935E75, vertexId=vertex_1452791806424_0001_1_01 [E501422D87154B5F84C18C62A15C5355]
2016-01-14 17:16:46,591 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:46,592 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: C0D7A89BDE404087BEAF7E8D55A48092 on vertex vertex_1452791806424_0001_1_00 [AF9B90FAA50E444D8556A417EAE7B3A7]
2016-01-14 17:16:46,592 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:46,592 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791806424_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E501422D87154B5F84C18C62A15C5355, vertexId=vertex_1452791806424_0001_1_01, initRequestedTime=1452791806585, initedTime=1452791806592, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:46,592 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791806424_0001_1_01 [E501422D87154B5F84C18C62A15C5355] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:46,592 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791806424_0001_1_00 [AF9B90FAA50E444D8556A417EAE7B3A7] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:46,592 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {C0D7A89BDE404087BEAF7E8D55A48092=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:46,593 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791806424_0001_1_00 [AF9B90FAA50E444D8556A417EAE7B3A7] parallelism set to 1
2016-01-14 17:16:46,593 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791806424_0001_1_00 [AF9B90FAA50E444D8556A417EAE7B3A7] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:46,593 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791806424_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791806424_0001_1_00, eventCount=1
2016-01-14 17:16:46,593 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:46,594 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791806424_0001_1][Event:VERTEX_INITIALIZED]: vertexName=AF9B90FAA50E444D8556A417EAE7B3A7, vertexId=vertex_1452791806424_0001_1_00, initRequestedTime=1452791806584, initedTime=1452791806593, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:46,594 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791806424_0001_1_00 [AF9B90FAA50E444D8556A417EAE7B3A7] with distanceFromRoot: 0
2016-01-14 17:16:46,594 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791806424_0001_1_00 [AF9B90FAA50E444D8556A417EAE7B3A7] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:46,594 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in AF9B90FAA50E444D8556A417EAE7B3A7
2016-01-14 17:16:46,594 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791806424_0001_1_00 [AF9B90FAA50E444D8556A417EAE7B3A7]
2016-01-14 17:16:46,594 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:46,594 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791806424_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791806424_0001_1_00, startRequestedTime=1452791806594, startedTime=1452791806594
2016-01-14 17:16:46,595 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791806424_0001_1_00 [AF9B90FAA50E444D8556A417EAE7B3A7] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:46,595 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:46,595 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791806424_0001_1][Event:TASK_STARTED]: vertexName=AF9B90FAA50E444D8556A417EAE7B3A7, taskId=task_1452791806424_0001_1_00_000000, scheduledTime=1452791806595, launchTime=1452791806595
2016-01-14 17:16:46,595 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791806424_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:46,595 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791806424_0001_1_00 for vertex: vertex_1452791806424_0001_1_01 [E501422D87154B5F84C18C62A15C5355] numStartedSources: 1 numSources: 1
2016-01-14 17:16:46,595 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791806424_0001_1_01 [E501422D87154B5F84C18C62A15C5355] with distanceFromRoot: 1
2016-01-14 17:16:46,595 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791806424_0001_1_00 [AF9B90FAA50E444D8556A417EAE7B3A7]
2016-01-14 17:16:46,596 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: AF9B90FAA50E444D8556A417EAE7B3A7 in vertex: E501422D87154B5F84C18C62A15C5355
2016-01-14 17:16:46,596 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: E501422D87154B5F84C18C62A15C5355 with 1 source tasks and 1 pending tasks
2016-01-14 17:16:46,596 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:46,596 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791806424_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791806424_0001_1_01, startRequestedTime=1452791806596, startedTime=1452791806596
2016-01-14 17:16:46,597 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791806424_0001_1_01 [E501422D87154B5F84C18C62A15C5355] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:46,597 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [5CC4023286854947A855F159EBADB99E/E0933C3C2A7D42638CB8D3E3023CCA8E] every/(1/1) ...tform/ALL_UNKNOWN_RESULTS, VertexName: AF9B90FAA50E444D8556A417EAE7B3A7, VertexParallelism: 1, TaskAttemptID:attempt_1452791806424_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C0D7A89BDE404087BEAF7E8D55A48092, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E501422D87154B5F84C18C62A15C5355, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:46,597 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791806424_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:46,597 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:46,598 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:46,598 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:46,598 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:46,599 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791806424_0001_00_000001
2016-01-14 17:16:46,599 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791806424_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:46,600 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791806424_0001_00_000001
2016-01-14 17:16:46,600 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:46,601 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791806424_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:46,601 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791806424_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791806424_0001_00_000001, launchTime=1452791806601
2016-01-14 17:16:46,601 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:46,602 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791806424_0001_00_000001
2016-01-14 17:16:46,602 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791806424_0001_1_00_000000_0] to container: [container_1452791806424_0001_00_000001]
2016-01-14 17:16:46,602 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791806424_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:46,603 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791806424_0001_00_000001 given task: attempt_1452791806424_0001_1_00_000000_0
2016-01-14 17:16:46,603 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791806424_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791806424_0001_1_00_000000_0
2016-01-14 17:16:46,603 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791806424_0001_1_00_000000_0] started. Is using containerId: [container_1452791806424_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:46 Starting to run new task attempt: attempt_1452791806424_0001_1_00_000000_0
2016-01-14 17:16:46,603 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791806424_0001_1_00_000000_0
2016-01-14 17:16:46,603 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:46,603 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:46,604 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:46,604 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:46,604 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:46,604 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:46,604 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [5CC4023286854947A855F159EBADB99E/E0933C3C2A7D42638CB8D3E3023CCA8E] every/(1/1) ...tform/ALL_UNKNOWN_RESULTS, VertexName: AF9B90FAA50E444D8556A417EAE7B3A7, VertexParallelism: 1, TaskAttemptID:attempt_1452791806424_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C0D7A89BDE404087BEAF7E8D55A48092, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E501422D87154B5F84C18C62A15C5355, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:46,604 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=717750272
2016-01-14 17:16:46,604 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:46,604 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791806424_0001, with dag id: dag_1452791806424_0001_1
2016-01-14 17:16:46,604 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791806424_0001_1_00_000000_0
2016-01-14 17:16:46,605 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:46,605 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:46,605 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791806424_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=AF9B90FAA50E444D8556A417EAE7B3A7, taskAttemptId=attempt_1452791806424_0001_1_00_000000_0, startTime=1452791806603, containerId=container_1452791806424_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791806424_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:46,605 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791806424_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:46,605 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791806424_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:46,606 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C0D7A89BDE404087BEAF7E8D55A48092, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:46,606 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:46,606 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C0D7A89BDE404087BEAF7E8D55A48092
2016-01-14 17:16:46,606 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: C0D7A89BDE404087BEAF7E8D55A48092
2016-01-14 17:16:46,607 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791806424_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:46,607 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:46,609 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:46,609 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: C0D7A89BDE404087BEAF7E8D55A48092
2016-01-14 17:16:46,609 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C0D7A89BDE404087BEAF7E8D55A48092
2016-01-14 17:16:46,610 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=E501422D87154B5F84C18C62A15C5355, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:46,610 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:46,610 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:46,610 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: E501422D87154B5F84C18C62A15C5355
2016-01-14 17:16:46,612 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:46,612 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: E501422D87154B5F84C18C62A15C5355
2016-01-14 17:16:46,613 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: E501422D87154B5F84C18C62A15C5355
2016-01-14 17:16:46,614 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: AF9B90FAA50E444D8556A417EAE7B3A7, ordinal: 0
2016-01-14 17:16:46,614 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AF9B90FAA50E444D8556A417EAE7B3A7, mem on start (mb), free: 310, total: 684, max: 684
2016-01-14 17:16:46,614 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:46,614 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:46,614 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:46,614 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:46,615 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:46,615 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:46,615 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:46,615 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:46,615 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:46,616 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 717750272, TotalAvailable: 480892682, TotalRequested/TotalJVMHeap:0.15
2016-01-14 17:16:46,616 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C0D7A89BDE404087BEAF7E8D55A48092, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:46,616 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, E501422D87154B5F84C18C62A15C5355, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:46,616 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:46,616 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C0D7A89BDE404087BEAF7E8D55A48092 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:46,616 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C0D7A89BDE404087BEAF7E8D55A48092
2016-01-14 17:16:46,617 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:46,617 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C0D7A89BDE404087BEAF7E8D55A48092
2016-01-14 17:16:46,616 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:46,617 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:46,617 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:46,617 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791806424_0001_1_00_000000_0
2016-01-14 17:16:46,618 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:46,618 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@2656bee
2016-01-14 17:16:46,619 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@2656bee
2016-01-14 17:16:46,619 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:46,619 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:46,620 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:46,621 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: C0D7A89BDE404087BEAF7E8D55A48092
2016-01-14 17:16:46,621 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 25DCC72E236C478BBE4C4B6AB5837C81
2016-01-14 17:16:46,621 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 25DCC72E236C478BBE4C4B6AB5837C81
2016-01-14 17:16:46,623 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@16e68af4; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@79aca4ea; comparator=cascading.tuple.hadoop.util.TupleComparator@357f3d0e; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:46,624 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:46,624 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:46,624 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:46,651 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:46,651 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:46,652 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:46,652 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:46,652 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:46,652 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:46,653 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:46,653 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: AF9B90FAA50E444D8556A417EAE7B3A7, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:46,656 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791806424_0001_1_00_000000_0
2016-01-14 17:16:46,656 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AF9B90FAA50E444D8556A417EAE7B3A7, mem on close (mb), free: 330, total: 684, max: 684
2016-01-14 17:16:46,656 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:46,656 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:46,656 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:46,657 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:46,659 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:46 Completed running task attempt: attempt_1452791806424_0001_1_00_000000_0
2016-01-14 17:16:46,662 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791806424_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:46,662 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=13, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=717750272, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791806653, Process_Duration=3, Process_End_Time=1452791806656, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:46,662 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791806424_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791806424_0001_1_00, eventCount=1
2016-01-14 17:16:46,663 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:46,663 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791806424_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=AF9B90FAA50E444D8556A417EAE7B3A7, taskAttemptId=attempt_1452791806424_0001_1_00_000000_0, startTime=1452791806603, finishTime=1452791806663, timeTaken=60, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=13, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=717750272, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791806653, Process_Duration=3, Process_End_Time=1452791806656, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:46,663 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791806424_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:46,663 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:46,662 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:46,663 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:46,663 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791806424_0001_1_00_000000_0
2016-01-14 17:16:46,664 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791806424_0001_1][Event:TASK_FINISHED]: vertexName=AF9B90FAA50E444D8556A417EAE7B3A7, taskId=task_1452791806424_0001_1_00_000000, startTime=1452791806603, finishTime=1452791806664, timeTaken=61, status=SUCCEEDED, successfulAttemptID=attempt_1452791806424_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=13, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=717750272, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791806653, Process_Duration=3, Process_End_Time=1452791806656, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:46,664 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791806424_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:46,664 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791806424_0001_1_00 [AF9B90FAA50E444D8556A417EAE7B3A7] : 1
2016-01-14 17:16:46,664 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791806424_0001_1_00 [AF9B90FAA50E444D8556A417EAE7B3A7]
2016-01-14 17:16:46,665 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791806424_0001_00_000001
2016-01-14 17:16:46,665 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791806424_0001_1][Event:VERTEX_FINISHED]: vertexName=AF9B90FAA50E444D8556A417EAE7B3A7, vertexId=vertex_1452791806424_0001_1_00, initRequestedTime=1452791806584, initedTime=1452791806593, startRequestedTime=1452791806594, startedTime=1452791806594, finishTime=1452791806664, timeTaken=70, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=13, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=717750272, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791806653, Process_Duration=3, Process_End_Time=1452791806656, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791806603, firstTasksToStart=[ task_1452791806424_0001_1_00_000000 ], lastTaskFinishTime=1452791806663, lastTasksToFinish=[ task_1452791806424_0001_1_00_000000 ], minTaskDuration=60, maxTaskDuration=60, avgTaskDuration=60.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791806424_0001_1_00_000000 ], longestDurationTasks=[ task_1452791806424_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:46,665 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:46,665 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791806424_0001_1_00 [AF9B90FAA50E444D8556A417EAE7B3A7] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:46,665 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791806424_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:46,665 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791806424_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791806424_0001_00_000001, stoppedTime=1452791806665, exitStatus=0
2016-01-14 17:16:46,665 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791806424_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:46,665 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:46,666 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791806424_0001_1_01 [E501422D87154B5F84C18C62A15C5355] attempt: attempt_1452791806424_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:46,666 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: E501422D87154B5F84C18C62A15C5355
2016-01-14 17:16:46,666 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791806424_0001_1_01 [E501422D87154B5F84C18C62A15C5355]
2016-01-14 17:16:46,666 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791806424_0001_1_00 [AF9B90FAA50E444D8556A417EAE7B3A7] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:46,666 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791806424_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:46,666 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:46,667 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791806424_0001_1][Event:TASK_STARTED]: vertexName=E501422D87154B5F84C18C62A15C5355, taskId=task_1452791806424_0001_1_01_000000, scheduledTime=1452791806667, launchTime=1452791806667
2016-01-14 17:16:46,667 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791806424_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:46,667 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:46,667 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791806424_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:46,667 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791806424_0001_1_01 [E501422D87154B5F84C18C62A15C5355]
2016-01-14 17:16:46,667 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [5CC4023286854947A855F159EBADB99E/E0933C3C2A7D42638CB8D3E3023CCA8E] every/(1/1) ...tform/ALL_UNKNOWN_RESULTS, VertexName: E501422D87154B5F84C18C62A15C5355, VertexParallelism: 1, TaskAttemptID:attempt_1452791806424_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=AF9B90FAA50E444D8556A417EAE7B3A7, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=EDCF24996B0141469E6153427F935E75, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:46,668 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791806424_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:46,668 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:46,668 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791806424_0001_00_000002
2016-01-14 17:16:46,668 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791806424_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:46,670 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791806424_0001_00_000002
2016-01-14 17:16:46,670 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:46,670 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791806424_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:46,670 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:46,670 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791806424_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791806424_0001_00_000002, launchTime=1452791806670
2016-01-14 17:16:46,671 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791806424_0001_00_000002
2016-01-14 17:16:46,671 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791806424_0001_1_01_000000_0] to container: [container_1452791806424_0001_00_000002]
2016-01-14 17:16:46,671 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791806424_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:46,671 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791806424_0001_00_000002 given task: attempt_1452791806424_0001_1_01_000000_0
2016-01-14 17:16:46,671 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791806424_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791806424_0001_1_01_000000_0
2016-01-14 17:16:46,672 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791806424_0001_1_01_000000_0] started. Is using containerId: [container_1452791806424_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:46,672 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791806424_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E501422D87154B5F84C18C62A15C5355, taskAttemptId=attempt_1452791806424_0001_1_01_000000_0, startTime=1452791806671, containerId=container_1452791806424_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791806424_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:46 Starting to run new task attempt: attempt_1452791806424_0001_1_01_000000_0
2016-01-14 17:16:46,672 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791806424_0001_1_01_000000_0
2016-01-14 17:16:46,672 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:46,672 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:46,672 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:46,672 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791806424_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:46,672 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791806424_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:46,672 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:46,673 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:46,673 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:46,673 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:46,673 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [5CC4023286854947A855F159EBADB99E/E0933C3C2A7D42638CB8D3E3023CCA8E] every/(1/1) ...tform/ALL_UNKNOWN_RESULTS, VertexName: E501422D87154B5F84C18C62A15C5355, VertexParallelism: 1, TaskAttemptID:attempt_1452791806424_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=AF9B90FAA50E444D8556A417EAE7B3A7, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=EDCF24996B0141469E6153427F935E75, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:46,674 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=717750272
2016-01-14 17:16:46,674 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791806424_0001_1_01_000000_0
2016-01-14 17:16:46,674 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:46,674 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:46,675 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791806424_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:46,675 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791806424_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:46,675 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=AF9B90FAA50E444D8556A417EAE7B3A7, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:46,675 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:46,675 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791806424_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:46,676 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791806424_0001_00_000001, Exiting
2016-01-14 17:16:46,676 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791806424_0001_00_000001
2016-01-14 17:16:46,676 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791806424_0001_00_000001
2016-01-14 17:16:46,676 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:46,676 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: AF9B90FAA50E444D8556A417EAE7B3A7
2016-01-14 17:16:46,677 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=EDCF24996B0141469E6153427F935E75, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:46,677 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:46,678 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: EDCF24996B0141469E6153427F935E75
2016-01-14 17:16:46,678 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:46,678 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: EDCF24996B0141469E6153427F935E75
2016-01-14 17:16:46,678 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:46,679 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 645975232, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:46,679 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:46,680 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 645975232, type: INPUT, componentVertexName: AF9B90FAA50E444D8556A417EAE7B3A7
2016-01-14 17:16:46,680 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: AF9B90FAA50E444D8556A417EAE7B3A7
2016-01-14 17:16:46,682 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E501422D87154B5F84C18C62A15C5355, ordinal: 1
2016-01-14 17:16:46,682 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E501422D87154B5F84C18C62A15C5355, mem on start (mb), free: 324, total: 684, max: 684
2016-01-14 17:16:46,682 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:46,683 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:46,683 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:46,683 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791806424_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:46,683 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791806424_0001_00_000001 completed successfully
2016-01-14 17:16:46,684 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791806424_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:46,689 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:46,689 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: EDCF24996B0141469E6153427F935E75
2016-01-14 17:16:46,689 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:46,689 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:46,689 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:46,690 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:46,690 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:46,690 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:46,690 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 645975232, TotalRequestedScaled: 5.962848295384616E8, TotalJVMHeap: 717750272, TotalAvailable: 480892682, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:46,690 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, EDCF24996B0141469E6153427F935E75, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:46,690 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, AF9B90FAA50E444D8556A417EAE7B3A7, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=645975232, allocated=480892682
2016-01-14 17:16:46,690 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:46,690 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: AF9B90FAA50E444D8556A417EAE7B3A7 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:46,690 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:46,690 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:46,690 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: AF9B90FAA50E444D8556A417EAE7B3A7
2016-01-14 17:16:46,691 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:46,691 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:46,691 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: AF9B90FAA50E444D8556A417EAE7B3A7 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:46,691 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=645975232, postMergeMem=0, RuntimeTotalAvailable=480892682. Updated to: ShuffleMem=480892682, postMergeMem: 0
2016-01-14 17:16:46,691 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=480892682, maxSingleShuffleLimit=120223168, mergeThreshold=432803392, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:46,692 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:46,693 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: AF9B90FAA50E444D8556A417EAE7B3A7
2016-01-14 17:16:46,693 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:46,693 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791806424_0001_1_01_000000_0
2016-01-14 17:16:46,694 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791806424_0001_1_00_000000_0_10267, runDuration: 50204, ]
2016-01-14 17:16:46,695 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791806424_0001_1_00_000000_0_10267] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791806695, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 17:16:46,695 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : AF9B90FAA50E444D8556A417EAE7B3A7
2016-01-14 17:16:46,695 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:46,695 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [AF9B90FAA50E444D8556A417EAE7B3A7] #45 in 1ms
2016-01-14 17:16:46,695 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [AF9B90FAA50E444D8556A417EAE7B3A7] #45, status:false, isInterrupted:false
2016-01-14 17:16:46,696 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:46,696 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:46,696 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:46,696 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:46,696 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:46,696 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 25DCC72E236C478BBE4C4B6AB5837C81
2016-01-14 17:16:46,696 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:46,697 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : AF9B90FAA50E444D8556A417EAE7B3A7
2016-01-14 17:16:46,697 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_RESULTS"], id: EDCF24996B0141469E6153427F935E75
2016-01-14 17:16:46,697 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_RESULTS"]
2016-01-14 17:16:46,697 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 25DCC72E236C478BBE4C4B6AB5837C81, for 1 inputs
2016-01-14 17:16:46,697 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E501422D87154B5F84C18C62A15C5355, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:46,698 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@380bdc4b; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@12abdcac
2016-01-14 17:16:46,697 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:46,701 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:46,701 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:46,701 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791806424_0001_1_01_000000_0
2016-01-14 17:16:46,702 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791806424_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:46,702 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791806424_0001_1_01_000000_0
2016-01-14 17:16:46,702 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E501422D87154B5F84C18C62A15C5355, mem on close (mb), free: 321, total: 684, max: 684
2016-01-14 17:16:46,702 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: AF9B90FAA50E444D8556A417EAE7B3A7
2016-01-14 17:16:46,702 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:46,703 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:46,703 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:46,703 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:46,703 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:46 Completed running task attempt: attempt_1452791806424_0001_1_01_000000_0
2016-01-14 17:16:46,703 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791806424_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:46,704 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791806424_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E501422D87154B5F84C18C62A15C5355, taskAttemptId=attempt_1452791806424_0001_1_01_000000_0, startTime=1452791806671, finishTime=1452791806704, timeTaken=33, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=652, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=717750272, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791806697, Process_Duration=5, Process_End_Time=1452791806702, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:46,704 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791806424_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:46,704 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:46,704 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:46,704 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791806424_0001_1_01_000000_0
2016-01-14 17:16:46,705 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:46,705 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791806424_0001_1][Event:TASK_FINISHED]: vertexName=E501422D87154B5F84C18C62A15C5355, taskId=task_1452791806424_0001_1_01_000000, startTime=1452791806671, finishTime=1452791806705, timeTaken=34, status=SUCCEEDED, successfulAttemptID=attempt_1452791806424_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=652, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=717750272, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791806697, Process_Duration=5, Process_End_Time=1452791806702, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:46,705 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791806424_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:46,705 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791806424_0001_1_01 [E501422D87154B5F84C18C62A15C5355] : 1
2016-01-14 17:16:46,705 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791806424_0001_1_01 [E501422D87154B5F84C18C62A15C5355]
2016-01-14 17:16:46,706 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:46,706 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791806424_0001_1][Event:VERTEX_FINISHED]: vertexName=E501422D87154B5F84C18C62A15C5355, vertexId=vertex_1452791806424_0001_1_01, initRequestedTime=1452791806585, initedTime=1452791806592, startRequestedTime=1452791806596, startedTime=1452791806596, finishTime=1452791806705, timeTaken=109, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=652, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=717750272, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791806697, Process_Duration=5, Process_End_Time=1452791806702, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452791806671, firstTasksToStart=[ task_1452791806424_0001_1_01_000000 ], lastTaskFinishTime=1452791806704, lastTasksToFinish=[ task_1452791806424_0001_1_01_000000 ], minTaskDuration=33, maxTaskDuration=33, avgTaskDuration=33.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791806424_0001_1_01_000000 ], longestDurationTasks=[ task_1452791806424_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:46,706 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791806424_0001_1_01 [E501422D87154B5F84C18C62A15C5355] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:46,706 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791806424_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:46,706 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=652, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=717750272, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791806697, Process_Duration=5, Process_End_Time=1452791806702, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 17:16:46,706 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791806424_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791806424_0001_00_000002, stoppedTime=1452791806706, exitStatus=0
2016-01-14 17:16:46,706 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791806424_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:46,707 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791806424_0001_1_01 [E501422D87154B5F84C18C62A15C5355] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:46,707 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791806424_0001_1
2016-01-14 17:16:46,707 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791806424_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452791806424_0001_1
2016-01-14 17:16:46,707 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: EDCF24996B0141469E6153427F935E75 for vertex: vertex_1452791806424_0001_1_01 [E501422D87154B5F84C18C62A15C5355]
2016-01-14 17:16:46,708 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:46,708 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:46,708 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791806424_0001_00_000002
2016-01-14 17:16:46,708 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791806424_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 17:16:46,708 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791806424_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:46,708 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791806424_0001_00_000002, Exiting
2016-01-14 17:16:46,708 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791806424_0001_00_000002
2016-01-14 17:16:46,708 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791806424_0001_00_000002
2016-01-14 17:16:46,709 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791806424_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:46,709 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791806424_0001_00_000002 completed successfully
2016-01-14 17:16:46,709 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:46,714 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452791806424_0001_1_00 [AF9B90FAA50E444D8556A417EAE7B3A7]
2016-01-14 17:16:46,716 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791806424_0001_1, queueSize=0
2016-01-14 17:16:46,716 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:46,717 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791806424_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791806424_0001_1, startTime=1452791806583, finishTime=1452791806714, timeTaken=131, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=1505, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=13, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1435500544, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905583613350, Process_Duration=8, Process_End_Time=2905583613358, Read_Duration=2, Tuples_Read=40, Tuples_Written=40, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 17:16:46,717 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791806424_0001_1 finished with state: SUCCEEDED
2016-01-14 17:16:46,717 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791806424_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:46,717 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452791806424_0001_00_000002
2016-01-14 17:16:46,718 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791806424_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:16:46 Completed Dag: dag_1452791806424_0001_1
2016-01-14 17:16:46,718 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791806424_0001_1, dagState=SUCCEEDED
2016-01-14 17:16:46,718 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791806424_0001_1_post
2016-01-14 17:16:46,719 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:46,719 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:46,723 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...tform/ALL_UNKNOWN_RESULTS, applicationId=application_1452791806424_0001
2016-01-14 17:16:46,724 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:46,724 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:46,725 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:46,725 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:46,725 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:46,725 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:46,725 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:46,726 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:46,726 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:46,726 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:46,726 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:46,727 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:46,727 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:46,730 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5CC4023286854947A855F159EBADB99E/E0933C3C2A7D42638CB8D3E3023CCA8E/.tez/application_1452791806424_0001
2016-01-14 17:16:46,731 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.323
2016-01-14 17:16:46,763 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 17:16:46,763 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 17:16:46,793 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_UNKNOWN_RESULTS
2016-01-14 17:16:46,813 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:46,813 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:46,840 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.005
2016-01-14 17:16:46,847 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-01-14 17:16:46,848 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:46,848 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:46,848 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:46,861 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:46,865 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_ALL
2016-01-14 17:16:46,904 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:46,904 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:46,904 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_ALL"]
2016-01-14 17:16:46,904 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:46,904 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:46,905 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:46,905 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...yplatform/ALL_UNKNOWN_ALL
2016-01-14 17:16:46,906 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:46,917 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:46,921 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:46,922 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:46,922 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:46,930 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/348F69053FE34C2DABDAB5CCCD50335E/F6AEF9483E194EDCBFCF28ED6BF6A2DC/.tez/application_1452791806921_0001 doesn't exist and is created
2016-01-14 17:16:46,946 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:46,946 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:46,946 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/348F69053FE34C2DABDAB5CCCD50335E/F6AEF9483E194EDCBFCF28ED6BF6A2DC/.tez/application_1452791806921_0001
2016-01-14 17:16:46,948 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791806921_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:46,957 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:46,957 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:46,958 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:46,958 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:46,958 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/348F69053FE34C2DABDAB5CCCD50335E/F6AEF9483E194EDCBFCF28ED6BF6A2DC/.tez/application_1452791806921_0001/localmode-log-dir
2016-01-14 17:16:46,958 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/348F69053FE34C2DABDAB5CCCD50335E/F6AEF9483E194EDCBFCF28ED6BF6A2DC/.tez/application_1452791806921_0001/localmode-log-dir/history.txt.appattempt_1452791806921_0001_000000, maxErrors=10
2016-01-14 17:16:46,959 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:46,959 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791806921_0001_000000, appSubmitTime=1452791806948, launchTime=1452791806948
2016-01-14 17:16:46,961 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:46,961 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:46,962 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:46,963 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:46,964 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:58400
2016-01-14 17:16:46,966 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:46,967 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791806921_0001_000000, startTime=1452791806967
2016-01-14 17:16:46,967 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:46,967 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:47,046 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:47,046 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:47,046 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...yplatform/ALL_UNKNOWN_ALL, applicationId=application_1452791806921_0001, dagName=[348F69053FE34C2DABDAB5CCCD50335E/F6AEF9483E194EDCBFCF28ED6BF6A2DC] every/(1/1) ...yplatform/ALL_UNKNOWN_ALL
2016-01-14 17:16:47,060 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [348F69053FE34C2DABDAB5CCCD50335E/F6AEF9483E194EDCBFCF28ED6BF6A2DC] every/(1/1) ...yplatform/ALL_UNKNOWN_ALL
2016-01-14 17:16:47,062 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791806921_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/348F69053FE34C2DABDAB5CCCD50335E/F6AEF9483E194EDCBFCF28ED6BF6A2DC/.tez/application_1452791806921_0001/localmode-log-dir/dag_1452791806921_0001_1.dot
2016-01-14 17:16:47,062 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/348F69053FE34C2DABDAB5CCCD50335E/F6AEF9483E194EDCBFCF28ED6BF6A2DC/.tez/application_1452791806921_0001/localmode-log-dir/dag_1452791806921_0001_1-tez-dag.pb.txt
2016-01-14 17:16:47,068 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791806921_0001_1
2016-01-14 17:16:47,068 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:47,068 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [348F69053FE34C2DABDAB5CCCD50335E/F6AEF9483E194EDCBFCF28ED6BF6A2DC] every/(1/1) ...yplatform/ALL_UNKNOWN_ALL
2016-01-14 17:16:47 Running Dag: dag_1452791806921_0001_1
2016-01-14 17:16:47,080 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791806921_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791806921_0001_1, submitTime=1452791807061
2016-01-14 17:16:47,080 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:47,080 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex F72BC7F2F39A4E34AAFA5DFA005B5660
2016-01-14 17:16:47,081 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:47,081 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791806921_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791806921_0001_1, initTime=1452791807080
2016-01-14 17:16:47,081 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:47,081 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791806921_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:47,081 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791806921_0001_1][Event:DAG_STARTED]: dagID=dag_1452791806921_0001_1, startTime=1452791807081
2016-01-14 17:16:47,081 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:47,082 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791806921_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:47,082 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: CD485A9F0B9F45ECB407739AAE80350A : {BA3A77A0B19B45B68D4A7D1B85304EAE={InputName=BA3A77A0B19B45B68D4A7D1B85304EAE}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:47,082 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: BA3A77A0B19B45B68D4A7D1B85304EAE, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:47,082 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791806921_0001_1_00 [CD485A9F0B9F45ECB407739AAE80350A]
2016-01-14 17:16:47,082 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791806921_0001_1_00 [CD485A9F0B9F45ECB407739AAE80350A]
2016-01-14 17:16:47,082 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791806921_0001_1_00 [CD485A9F0B9F45ECB407739AAE80350A]
2016-01-14 17:16:47,082 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791806921_0001_1_00 [CD485A9F0B9F45ECB407739AAE80350A]. Starting root input initializers: 1
2016-01-14 17:16:47,082 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...yplatform/ALL_UNKNOWN_ALL, applicationId=application_1452791806921_0001, dagName=[348F69053FE34C2DABDAB5CCCD50335E/F6AEF9483E194EDCBFCF28ED6BF6A2DC] every/(1/1) ...yplatform/ALL_UNKNOWN_ALL
2016-01-14 17:16:47,083 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:47,085 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791806921_0001_1_00 [CD485A9F0B9F45ECB407739AAE80350A] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:47,085 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791806921_0001_1_01 [F72BC7F2F39A4E34AAFA5DFA005B5660]
2016-01-14 17:16:47,087 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: BA3A77A0B19B45B68D4A7D1B85304EAE on vertex vertex_1452791806921_0001_1_00 [CD485A9F0B9F45ECB407739AAE80350A]
2016-01-14 17:16:47,088 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:47,088 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791806921_0001_1_01 [F72BC7F2F39A4E34AAFA5DFA005B5660]
2016-01-14 17:16:47,088 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791806921_0001_1_01 [F72BC7F2F39A4E34AAFA5DFA005B5660]
2016-01-14 17:16:47,088 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791806921_0001_1_01 [F72BC7F2F39A4E34AAFA5DFA005B5660]
2016-01-14 17:16:47,088 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=B11C88130CF142728956CA44EEEBD677, vertexId=vertex_1452791806921_0001_1_01 [F72BC7F2F39A4E34AAFA5DFA005B5660], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:47,089 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=B11C88130CF142728956CA44EEEBD677, vertexId=vertex_1452791806921_0001_1_01 [F72BC7F2F39A4E34AAFA5DFA005B5660]
2016-01-14 17:16:47,089 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:47,089 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input BA3A77A0B19B45B68D4A7D1B85304EAE asking for 1 tasks. Headroom: 684 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:47,089 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:47,089 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:47,090 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=B11C88130CF142728956CA44EEEBD677, vertexName=F72BC7F2F39A4E34AAFA5DFA005B5660, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:47,090 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=B11C88130CF142728956CA44EEEBD677, vertexId=vertex_1452791806921_0001_1_01 [F72BC7F2F39A4E34AAFA5DFA005B5660]
2016-01-14 17:16:47,090 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:47,091 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: BA3A77A0B19B45B68D4A7D1B85304EAE on vertex vertex_1452791806921_0001_1_00 [CD485A9F0B9F45ECB407739AAE80350A]
2016-01-14 17:16:47,091 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:47,092 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791806921_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F72BC7F2F39A4E34AAFA5DFA005B5660, vertexId=vertex_1452791806921_0001_1_01, initRequestedTime=1452791807085, initedTime=1452791807091, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:47,092 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791806921_0001_1_01 [F72BC7F2F39A4E34AAFA5DFA005B5660] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:47,092 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791806921_0001_1_00 [CD485A9F0B9F45ECB407739AAE80350A] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:47,092 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {BA3A77A0B19B45B68D4A7D1B85304EAE=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:47,092 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791806921_0001_1_00 [CD485A9F0B9F45ECB407739AAE80350A] parallelism set to 1
2016-01-14 17:16:47,092 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791806921_0001_1_00 [CD485A9F0B9F45ECB407739AAE80350A] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:47,092 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791806921_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791806921_0001_1_00, eventCount=1
2016-01-14 17:16:47,093 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:47,093 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791806921_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CD485A9F0B9F45ECB407739AAE80350A, vertexId=vertex_1452791806921_0001_1_00, initRequestedTime=1452791807082, initedTime=1452791807093, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:47,093 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791806921_0001_1_00 [CD485A9F0B9F45ECB407739AAE80350A] with distanceFromRoot: 0
2016-01-14 17:16:47,093 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791806921_0001_1_00 [CD485A9F0B9F45ECB407739AAE80350A] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:47,093 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in CD485A9F0B9F45ECB407739AAE80350A
2016-01-14 17:16:47,094 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791806921_0001_1_00 [CD485A9F0B9F45ECB407739AAE80350A]
2016-01-14 17:16:47,094 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:47,094 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791806921_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791806921_0001_1_00, startRequestedTime=1452791807093, startedTime=1452791807093
2016-01-14 17:16:47,094 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791806921_0001_1_00 [CD485A9F0B9F45ECB407739AAE80350A] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:47,095 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:47,095 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791806921_0001_1][Event:TASK_STARTED]: vertexName=CD485A9F0B9F45ECB407739AAE80350A, taskId=task_1452791806921_0001_1_00_000000, scheduledTime=1452791807094, launchTime=1452791807094
2016-01-14 17:16:47,095 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791806921_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:47,095 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791806921_0001_1_00 for vertex: vertex_1452791806921_0001_1_01 [F72BC7F2F39A4E34AAFA5DFA005B5660] numStartedSources: 1 numSources: 1
2016-01-14 17:16:47,095 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791806921_0001_1_01 [F72BC7F2F39A4E34AAFA5DFA005B5660] with distanceFromRoot: 1
2016-01-14 17:16:47,095 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791806921_0001_1_00 [CD485A9F0B9F45ECB407739AAE80350A]
2016-01-14 17:16:47,096 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: CD485A9F0B9F45ECB407739AAE80350A in vertex: F72BC7F2F39A4E34AAFA5DFA005B5660
2016-01-14 17:16:47,096 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: F72BC7F2F39A4E34AAFA5DFA005B5660 with 1 source tasks and 1 pending tasks
2016-01-14 17:16:47,096 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:47,096 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791806921_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791806921_0001_1_01, startRequestedTime=1452791807096, startedTime=1452791807096
2016-01-14 17:16:47,096 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791806921_0001_1_01 [F72BC7F2F39A4E34AAFA5DFA005B5660] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:47,097 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [348F69053FE34C2DABDAB5CCCD50335E/F6AEF9483E194EDCBFCF28ED6BF6A2DC] every/(1/1) ...yplatform/ALL_UNKNOWN_ALL, VertexName: CD485A9F0B9F45ECB407739AAE80350A, VertexParallelism: 1, TaskAttemptID:attempt_1452791806921_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BA3A77A0B19B45B68D4A7D1B85304EAE, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F72BC7F2F39A4E34AAFA5DFA005B5660, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:47,097 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791806921_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:47,097 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:47,097 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791806921_0001, with dag id: dag_1452791806921_0001_1
2016-01-14 17:16:47,098 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:47,098 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:47,098 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:47,098 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791806921_0001_00_000001
2016-01-14 17:16:47,098 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791806921_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:47,100 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791806921_0001_00_000001
2016-01-14 17:16:47,100 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:47,100 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791806921_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:47,100 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:47,101 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791806921_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791806921_0001_00_000001, launchTime=1452791807100
2016-01-14 17:16:47,101 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791806921_0001_00_000001
2016-01-14 17:16:47,102 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791806921_0001_1_00_000000_0] to container: [container_1452791806921_0001_00_000001]
2016-01-14 17:16:47,102 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791806921_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:47,102 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791806921_0001_1_00_000000_0] started. Is using containerId: [container_1452791806921_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:47,102 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791806921_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CD485A9F0B9F45ECB407739AAE80350A, taskAttemptId=attempt_1452791806921_0001_1_00_000000_0, startTime=1452791807102, containerId=container_1452791806921_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791806921_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:47,102 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791806921_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:47,102 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:47,102 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791806921_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:47,103 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791806921_0001_00_000001 given task: attempt_1452791806921_0001_1_00_000000_0
2016-01-14 17:16:47,103 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791806921_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791806921_0001_1_00_000000_0
2016-01-14 17:16:47 Starting to run new task attempt: attempt_1452791806921_0001_1_00_000000_0
2016-01-14 17:16:47,103 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791806921_0001_1_00_000000_0
2016-01-14 17:16:47,103 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:47,103 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:47,104 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:47,104 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:47,104 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:47,104 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:47,104 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [348F69053FE34C2DABDAB5CCCD50335E/F6AEF9483E194EDCBFCF28ED6BF6A2DC] every/(1/1) ...yplatform/ALL_UNKNOWN_ALL, VertexName: CD485A9F0B9F45ECB407739AAE80350A, VertexParallelism: 1, TaskAttemptID:attempt_1452791806921_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BA3A77A0B19B45B68D4A7D1B85304EAE, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F72BC7F2F39A4E34AAFA5DFA005B5660, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:47,104 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=717750272
2016-01-14 17:16:47,105 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791806921_0001_1_00_000000_0
2016-01-14 17:16:47,105 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:47,105 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:47,105 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=BA3A77A0B19B45B68D4A7D1B85304EAE, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:47,123 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:47,122 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F72BC7F2F39A4E34AAFA5DFA005B5660, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:47,105 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:47,123 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:47,124 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F72BC7F2F39A4E34AAFA5DFA005B5660
2016-01-14 17:16:47,123 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791806921_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:47,123 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: BA3A77A0B19B45B68D4A7D1B85304EAE
2016-01-14 17:16:47,126 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: BA3A77A0B19B45B68D4A7D1B85304EAE
2016-01-14 17:16:47,126 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:47,126 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:47,127 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: F72BC7F2F39A4E34AAFA5DFA005B5660
2016-01-14 17:16:47,127 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F72BC7F2F39A4E34AAFA5DFA005B5660
2016-01-14 17:16:47,127 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:47,127 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: BA3A77A0B19B45B68D4A7D1B85304EAE
2016-01-14 17:16:47,127 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: BA3A77A0B19B45B68D4A7D1B85304EAE
2016-01-14 17:16:47,129 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CD485A9F0B9F45ECB407739AAE80350A, ordinal: 0
2016-01-14 17:16:47,129 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CD485A9F0B9F45ECB407739AAE80350A, mem on start (mb), free: 431, total: 686, max: 686
2016-01-14 17:16:47,129 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:47,129 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:47,129 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:47,129 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:47,129 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:47,129 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:47,129 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:47,129 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:47,129 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:47,129 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 719847424, TotalAvailable: 480892682, TotalRequested/TotalJVMHeap:0.15
2016-01-14 17:16:47,129 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, BA3A77A0B19B45B68D4A7D1B85304EAE, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:47,129 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F72BC7F2F39A4E34AAFA5DFA005B5660, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:47,130 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:47,130 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: BA3A77A0B19B45B68D4A7D1B85304EAE being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:47,130 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:47,130 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:47,130 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: BA3A77A0B19B45B68D4A7D1B85304EAE
2016-01-14 17:16:47,130 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:47,130 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: BA3A77A0B19B45B68D4A7D1B85304EAE
2016-01-14 17:16:47,130 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:47,130 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791806921_0001_1_00_000000_0
2016-01-14 17:16:47,131 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:47,131 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@6deeba4a
2016-01-14 17:16:47,132 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@6deeba4a
2016-01-14 17:16:47,132 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:47,132 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:47,133 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:47,133 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: BA3A77A0B19B45B68D4A7D1B85304EAE
2016-01-14 17:16:47,133 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 76E14C63116E46C6965A296ABCBA0E46
2016-01-14 17:16:47,134 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 76E14C63116E46C6965A296ABCBA0E46
2016-01-14 17:16:47,134 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@423c8a44; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@14999855; comparator=cascading.tuple.hadoop.util.TupleComparator@6fdfff03; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:47,134 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:47,134 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:47,134 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:47,148 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:47,148 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:47,148 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:47,148 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:47,148 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:47,149 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:47,149 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:47,149 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CD485A9F0B9F45ECB407739AAE80350A, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:47,152 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791806921_0001_1_00_000000_0
2016-01-14 17:16:47,152 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CD485A9F0B9F45ECB407739AAE80350A, mem on close (mb), free: 329, total: 686, max: 686
2016-01-14 17:16:47,152 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:47,152 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:47,152 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:47,152 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:47,154 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:47 Completed running task attempt: attempt_1452791806921_0001_1_00_000000_0
2016-01-14 17:16:47,157 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791806921_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:47,157 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791806921_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791806921_0001_1_00, eventCount=1
2016-01-14 17:16:47,157 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=13, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=719847424, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791807149, Process_Duration=2, Process_End_Time=1452791807151, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:47,158 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:47,158 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791806921_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CD485A9F0B9F45ECB407739AAE80350A, taskAttemptId=attempt_1452791806921_0001_1_00_000000_0, startTime=1452791807102, finishTime=1452791807157, timeTaken=55, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=13, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=719847424, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791807149, Process_Duration=2, Process_End_Time=1452791807151, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:47,158 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791806921_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:47,158 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791806921_0001_1_00_000000_0
2016-01-14 17:16:47,158 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:47,158 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:47,158 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791806921_0001_1][Event:TASK_FINISHED]: vertexName=CD485A9F0B9F45ECB407739AAE80350A, taskId=task_1452791806921_0001_1_00_000000, startTime=1452791807102, finishTime=1452791807158, timeTaken=56, status=SUCCEEDED, successfulAttemptID=attempt_1452791806921_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=13, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=719847424, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791807149, Process_Duration=2, Process_End_Time=1452791807151, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:47,159 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791806921_0001_00_000001
2016-01-14 17:16:47,159 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791806921_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:47,159 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791806921_0001_1_00 [CD485A9F0B9F45ECB407739AAE80350A] : 1
2016-01-14 17:16:47,159 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791806921_0001_1_00 [CD485A9F0B9F45ECB407739AAE80350A]
2016-01-14 17:16:47,159 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:47,159 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:47,159 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791806921_0001_1][Event:VERTEX_FINISHED]: vertexName=CD485A9F0B9F45ECB407739AAE80350A, vertexId=vertex_1452791806921_0001_1_00, initRequestedTime=1452791807082, initedTime=1452791807093, startRequestedTime=1452791807093, startedTime=1452791807093, finishTime=1452791807159, timeTaken=66, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=13, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=719847424, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791807149, Process_Duration=2, Process_End_Time=1452791807151, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791807102, firstTasksToStart=[ task_1452791806921_0001_1_00_000000 ], lastTaskFinishTime=1452791807157, lastTasksToFinish=[ task_1452791806921_0001_1_00_000000 ], minTaskDuration=55, maxTaskDuration=55, avgTaskDuration=55.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791806921_0001_1_00_000000 ], longestDurationTasks=[ task_1452791806921_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:47,160 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791806921_0001_1_00 [CD485A9F0B9F45ECB407739AAE80350A] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:47,160 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791806921_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:47,160 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791806921_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791806921_0001_00_000001, stoppedTime=1452791807160, exitStatus=0
2016-01-14 17:16:47,160 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791806921_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:47,160 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791806921_0001_1_01 [F72BC7F2F39A4E34AAFA5DFA005B5660] attempt: attempt_1452791806921_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:47,160 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: F72BC7F2F39A4E34AAFA5DFA005B5660
2016-01-14 17:16:47,160 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791806921_0001_1_01 [F72BC7F2F39A4E34AAFA5DFA005B5660]
2016-01-14 17:16:47,161 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791806921_0001_1_00 [CD485A9F0B9F45ECB407739AAE80350A] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:47,161 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791806921_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:47,161 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791806921_0001_1][Event:TASK_STARTED]: vertexName=F72BC7F2F39A4E34AAFA5DFA005B5660, taskId=task_1452791806921_0001_1_01_000000, scheduledTime=1452791807161, launchTime=1452791807161
2016-01-14 17:16:47,161 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791806921_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:47,161 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791806921_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:47,161 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791806921_0001_1_01 [F72BC7F2F39A4E34AAFA5DFA005B5660]
2016-01-14 17:16:47,161 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [348F69053FE34C2DABDAB5CCCD50335E/F6AEF9483E194EDCBFCF28ED6BF6A2DC] every/(1/1) ...yplatform/ALL_UNKNOWN_ALL, VertexName: F72BC7F2F39A4E34AAFA5DFA005B5660, VertexParallelism: 1, TaskAttemptID:attempt_1452791806921_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=CD485A9F0B9F45ECB407739AAE80350A, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=B11C88130CF142728956CA44EEEBD677, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:47,161 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791806921_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:47,160 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:47,160 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791806921_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:47,162 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791806921_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:47,162 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791806921_0001_00_000001, Exiting
2016-01-14 17:16:47,162 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791806921_0001_00_000001
2016-01-14 17:16:47,162 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791806921_0001_00_000001
2016-01-14 17:16:47,162 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:47,162 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:47,162 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:47,163 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791806921_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:47,163 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791806921_0001_00_000001 completed successfully
2016-01-14 17:16:47,163 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791806921_0001_00_000002
2016-01-14 17:16:47,163 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791806921_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:47,165 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791806921_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:47,165 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791806921_0001_00_000002
2016-01-14 17:16:47,166 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 17:16:47,168 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791806921_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:47,168 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:47,168 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791806921_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791806921_0001_00_000002, launchTime=1452791807168
2016-01-14 17:16:47,169 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791806921_0001_00_000002
2016-01-14 17:16:47,169 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791806921_0001_1_01_000000_0] to container: [container_1452791806921_0001_00_000002]
2016-01-14 17:16:47,169 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791806921_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:47,170 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791806921_0001_1_01_000000_0] started. Is using containerId: [container_1452791806921_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:47,170 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:47,170 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791806921_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F72BC7F2F39A4E34AAFA5DFA005B5660, taskAttemptId=attempt_1452791806921_0001_1_01_000000_0, startTime=1452791807170, containerId=container_1452791806921_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791806921_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:47,170 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791806921_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:47,170 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791806921_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:47,170 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791806921_0001_00_000002 given task: attempt_1452791806921_0001_1_01_000000_0
2016-01-14 17:16:47,171 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791806921_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791806921_0001_1_01_000000_0
2016-01-14 17:16:47 Starting to run new task attempt: attempt_1452791806921_0001_1_01_000000_0
2016-01-14 17:16:47,171 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791806921_0001_1_01_000000_0
2016-01-14 17:16:47,171 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:47,171 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:47,172 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:47,172 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:47,172 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:47,172 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:47,173 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [348F69053FE34C2DABDAB5CCCD50335E/F6AEF9483E194EDCBFCF28ED6BF6A2DC] every/(1/1) ...yplatform/ALL_UNKNOWN_ALL, VertexName: F72BC7F2F39A4E34AAFA5DFA005B5660, VertexParallelism: 1, TaskAttemptID:attempt_1452791806921_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=CD485A9F0B9F45ECB407739AAE80350A, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=B11C88130CF142728956CA44EEEBD677, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:47,173 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=719847424
2016-01-14 17:16:47,173 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791806921_0001_1_01_000000_0
2016-01-14 17:16:47,173 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:47,173 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791806921_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:47,173 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:47,174 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:47,174 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=CD485A9F0B9F45ECB407739AAE80350A, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:47,174 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=B11C88130CF142728956CA44EEEBD677, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:47,175 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:47,175 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: CD485A9F0B9F45ECB407739AAE80350A
2016-01-14 17:16:47,175 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:47,176 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 647862656, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:47,177 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:47,177 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: B11C88130CF142728956CA44EEEBD677
2016-01-14 17:16:47,177 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:47,177 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: B11C88130CF142728956CA44EEEBD677
2016-01-14 17:16:47,177 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 647862656, type: INPUT, componentVertexName: CD485A9F0B9F45ECB407739AAE80350A
2016-01-14 17:16:47,178 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: CD485A9F0B9F45ECB407739AAE80350A
2016-01-14 17:16:47,178 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:47,182 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F72BC7F2F39A4E34AAFA5DFA005B5660, ordinal: 1
2016-01-14 17:16:47,183 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F72BC7F2F39A4E34AAFA5DFA005B5660, mem on start (mb), free: 324, total: 686, max: 686
2016-01-14 17:16:47,183 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:47,183 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:47,183 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:47,187 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:47,187 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: B11C88130CF142728956CA44EEEBD677
2016-01-14 17:16:47,187 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:47,187 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:47,188 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:47,188 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:47,188 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:47,189 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:47,189 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 647862656, TotalRequestedScaled: 5.980270670769231E8, TotalJVMHeap: 719847424, TotalAvailable: 482297774, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:47,189 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, B11C88130CF142728956CA44EEEBD677, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:47,189 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, CD485A9F0B9F45ECB407739AAE80350A, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=647862656, allocated=482297774
2016-01-14 17:16:47,189 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:47,190 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: CD485A9F0B9F45ECB407739AAE80350A being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:47,190 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: CD485A9F0B9F45ECB407739AAE80350A
2016-01-14 17:16:47,190 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:47,190 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:47,191 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: CD485A9F0B9F45ECB407739AAE80350A with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:47,191 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:47,191 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:47,191 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=647862656, postMergeMem=0, RuntimeTotalAvailable=482297774. Updated to: ShuffleMem=482297774, postMergeMem: 0
2016-01-14 17:16:47,191 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=482297774, maxSingleShuffleLimit=120574440, mergeThreshold=434067968, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:47,191 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:47,192 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: CD485A9F0B9F45ECB407739AAE80350A
2016-01-14 17:16:47,192 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:47,192 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791806921_0001_1_01_000000_0
2016-01-14 17:16:47,193 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791806921_0001_1_00_000000_0_10273, runDuration: 31689, ]
2016-01-14 17:16:47,194 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791806921_0001_1_00_000000_0_10273] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791807194, TimeTaken=0, Rate=0.00 MB/s
2016-01-14 17:16:47,194 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : CD485A9F0B9F45ECB407739AAE80350A
2016-01-14 17:16:47,195 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:47,195 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:47,195 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [CD485A9F0B9F45ECB407739AAE80350A] #46 in 1ms
2016-01-14 17:16:47,195 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 76E14C63116E46C6965A296ABCBA0E46
2016-01-14 17:16:47,196 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [CD485A9F0B9F45ECB407739AAE80350A] #46, status:false, isInterrupted:false
2016-01-14 17:16:47,196 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_ALL"], id: B11C88130CF142728956CA44EEEBD677
2016-01-14 17:16:47,196 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_ALL"]
2016-01-14 17:16:47,196 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 76E14C63116E46C6965A296ABCBA0E46, for 1 inputs
2016-01-14 17:16:47,197 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:47,197 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:47,197 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:47,197 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:47,197 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:47,197 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : CD485A9F0B9F45ECB407739AAE80350A
2016-01-14 17:16:47,197 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:47,198 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F72BC7F2F39A4E34AAFA5DFA005B5660, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:47,198 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@4fac1d53; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@2cb95d02
2016-01-14 17:16:47,201 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:47,202 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:47,202 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791806921_0001_1_01_000000_0
2016-01-14 17:16:47,202 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791806921_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:47,203 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791806921_0001_1_01_000000_0
2016-01-14 17:16:47,203 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F72BC7F2F39A4E34AAFA5DFA005B5660, mem on close (mb), free: 320, total: 686, max: 686
2016-01-14 17:16:47,203 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: CD485A9F0B9F45ECB407739AAE80350A
2016-01-14 17:16:47,203 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:47,204 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:47,204 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:47,204 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:47,204 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:47 Completed running task attempt: attempt_1452791806921_0001_1_01_000000_0
2016-01-14 17:16:47,205 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791806921_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:47,205 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=719847424, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791807198, Process_Duration=5, Process_End_Time=1452791807203, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 17:16:47,205 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791806921_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F72BC7F2F39A4E34AAFA5DFA005B5660, taskAttemptId=attempt_1452791806921_0001_1_01_000000_0, startTime=1452791807170, finishTime=1452791807205, timeTaken=35, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=719847424, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791807198, Process_Duration=5, Process_End_Time=1452791807203, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:47,205 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:47,206 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791806921_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:47,206 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791806921_0001_1_01_000000_0
2016-01-14 17:16:47,206 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:47,206 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:47,206 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791806921_0001_1][Event:TASK_FINISHED]: vertexName=F72BC7F2F39A4E34AAFA5DFA005B5660, taskId=task_1452791806921_0001_1_01_000000, startTime=1452791807170, finishTime=1452791807206, timeTaken=36, status=SUCCEEDED, successfulAttemptID=attempt_1452791806921_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=719847424, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791807198, Process_Duration=5, Process_End_Time=1452791807203, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:47,206 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791806921_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:47,207 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791806921_0001_1_01 [F72BC7F2F39A4E34AAFA5DFA005B5660] : 1
2016-01-14 17:16:47,207 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791806921_0001_1_01 [F72BC7F2F39A4E34AAFA5DFA005B5660]
2016-01-14 17:16:47,207 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791806921_0001_1][Event:VERTEX_FINISHED]: vertexName=F72BC7F2F39A4E34AAFA5DFA005B5660, vertexId=vertex_1452791806921_0001_1_01, initRequestedTime=1452791807085, initedTime=1452791807091, startRequestedTime=1452791807096, startedTime=1452791807096, finishTime=1452791807207, timeTaken=111, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=719847424, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791807198, Process_Duration=5, Process_End_Time=1452791807203, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452791807170, firstTasksToStart=[ task_1452791806921_0001_1_01_000000 ], lastTaskFinishTime=1452791807205, lastTasksToFinish=[ task_1452791806921_0001_1_01_000000 ], minTaskDuration=35, maxTaskDuration=35, avgTaskDuration=35.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791806921_0001_1_01_000000 ], longestDurationTasks=[ task_1452791806921_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:47,207 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791806921_0001_1_01 [F72BC7F2F39A4E34AAFA5DFA005B5660] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:47,207 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791806921_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:47,208 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791806921_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791806921_0001_00_000002, stoppedTime=1452791807207, exitStatus=0
2016-01-14 17:16:47,208 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791806921_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:47,208 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791806921_0001_1_01 [F72BC7F2F39A4E34AAFA5DFA005B5660] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:47,208 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791806921_0001_1
2016-01-14 17:16:47,208 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791806921_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452791806921_0001_1
2016-01-14 17:16:47,208 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: B11C88130CF142728956CA44EEEBD677 for vertex: vertex_1452791806921_0001_1_01 [F72BC7F2F39A4E34AAFA5DFA005B5660]
2016-01-14 17:16:47,209 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:47,209 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:47,210 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:47,210 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:47,210 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791806921_0001_00_000002
2016-01-14 17:16:47,210 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791806921_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 17:16:47,210 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791806921_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:47,211 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791806921_0001_00_000002, Exiting
2016-01-14 17:16:47,211 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791806921_0001_00_000002
2016-01-14 17:16:47,211 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791806921_0001_00_000002
2016-01-14 17:16:47,211 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791806921_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:47,211 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791806921_0001_00_000002 completed successfully
2016-01-14 17:16:47,219 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452791806921_0001_1_00 [CD485A9F0B9F45ECB407739AAE80350A]
2016-01-14 17:16:47,220 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791806921_0001_1, queueSize=0
2016-01-14 17:16:47,220 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:47,220 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791806921_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791806921_0001_1, startTime=1452791807081, finishTime=1452791807219, timeTaken=138, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=571, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=13, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1439694848, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905583614347, Process_Duration=7, Process_End_Time=2905583614354, Read_Duration=3, Tuples_Read=40, Tuples_Written=40, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 17:16:47,220 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791806921_0001_1 finished with state: SUCCEEDED
2016-01-14 17:16:47,220 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791806921_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:47,221 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452791806921_0001_00_000002
2016-01-14 17:16:47,221 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791806921_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:16:47 Completed Dag: dag_1452791806921_0001_1
2016-01-14 17:16:47,223 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791806921_0001_1, dagState=SUCCEEDED
2016-01-14 17:16:47,223 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791806921_0001_1_post
2016-01-14 17:16:47,224 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:47,224 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:47,227 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...yplatform/ALL_UNKNOWN_ALL, applicationId=application_1452791806921_0001
2016-01-14 17:16:47,228 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:47,228 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:47,228 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:47,229 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:47,229 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:47,229 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:47,230 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:47,230 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:47,230 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:47,230 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:47,230 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:47,231 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:47,231 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:47,234 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/348F69053FE34C2DABDAB5CCCD50335E/F6AEF9483E194EDCBFCF28ED6BF6A2DC/.tez/application_1452791806921_0001
2016-01-14 17:16:47,235 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.330
2016-01-14 17:16:47,296 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_UNKNOWN_ALL
2016-01-14 17:16:47,321 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:47,321 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:47,365 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.008
2016-01-14 17:16:47,372 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-01-14 17:16:47,372 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:47,372 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:47,372 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:47,394 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:47,400 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_REPLACE
2016-01-14 17:16:47,443 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 17:16:47,443 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 17:16:47,457 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:47,457 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:47,457 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_REPLACE"]
2016-01-14 17:16:47,458 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:47,458 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:47,460 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:47,460 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...tform/ALL_UNKNOWN_REPLACE
2016-01-14 17:16:47,460 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:47,471 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:47,475 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:47,476 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:47,476 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:47,484 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A27728F4E5C54B3FA02BAE87E6422171/D9768DF31E694AC681A71B3FC6F8AACF/.tez/application_1452791807475_0001 doesn't exist and is created
2016-01-14 17:16:47,500 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:47,500 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:47,501 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A27728F4E5C54B3FA02BAE87E6422171/D9768DF31E694AC681A71B3FC6F8AACF/.tez/application_1452791807475_0001
2016-01-14 17:16:47,503 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791807475_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:47,512 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:47,513 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:47,513 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:47,514 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:47,514 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A27728F4E5C54B3FA02BAE87E6422171/D9768DF31E694AC681A71B3FC6F8AACF/.tez/application_1452791807475_0001/localmode-log-dir
2016-01-14 17:16:47,514 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A27728F4E5C54B3FA02BAE87E6422171/D9768DF31E694AC681A71B3FC6F8AACF/.tez/application_1452791807475_0001/localmode-log-dir/history.txt.appattempt_1452791807475_0001_000000, maxErrors=10
2016-01-14 17:16:47,514 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:47,514 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791807475_0001_000000, appSubmitTime=1452791807502, launchTime=1452791807502
2016-01-14 17:16:47,518 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:49062
2016-01-14 17:16:47,519 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:47,519 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:47,520 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:47,521 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:47,522 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:47,522 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791807475_0001_000000, startTime=1452791807522
2016-01-14 17:16:47,523 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:47,523 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:47,601 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:47,601 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:47,601 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...tform/ALL_UNKNOWN_REPLACE, applicationId=application_1452791807475_0001, dagName=[A27728F4E5C54B3FA02BAE87E6422171/D9768DF31E694AC681A71B3FC6F8AACF] every/(1/1) ...tform/ALL_UNKNOWN_REPLACE
2016-01-14 17:16:47,615 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [A27728F4E5C54B3FA02BAE87E6422171/D9768DF31E694AC681A71B3FC6F8AACF] every/(1/1) ...tform/ALL_UNKNOWN_REPLACE
2016-01-14 17:16:47,616 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791807475_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A27728F4E5C54B3FA02BAE87E6422171/D9768DF31E694AC681A71B3FC6F8AACF/.tez/application_1452791807475_0001/localmode-log-dir/dag_1452791807475_0001_1.dot
2016-01-14 17:16:47,616 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A27728F4E5C54B3FA02BAE87E6422171/D9768DF31E694AC681A71B3FC6F8AACF/.tez/application_1452791807475_0001/localmode-log-dir/dag_1452791807475_0001_1-tez-dag.pb.txt
2016-01-14 17:16:47,622 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791807475_0001_1
2016-01-14 17:16:47,622 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:47,622 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [A27728F4E5C54B3FA02BAE87E6422171/D9768DF31E694AC681A71B3FC6F8AACF] every/(1/1) ...tform/ALL_UNKNOWN_REPLACE
2016-01-14 17:16:47 Running Dag: dag_1452791807475_0001_1
2016-01-14 17:16:47,632 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:47,633 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791807475_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791807475_0001_1, submitTime=1452791807615
2016-01-14 17:16:47,633 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex B27AFB27853E4D72B3907EB9F4A97A56
2016-01-14 17:16:47,633 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:47,633 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:47,634 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791807475_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791807475_0001_1, initTime=1452791807633
2016-01-14 17:16:47,634 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791807475_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:47,634 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791807475_0001_1][Event:DAG_STARTED]: dagID=dag_1452791807475_0001_1, startTime=1452791807634
2016-01-14 17:16:47,634 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:47,634 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...tform/ALL_UNKNOWN_REPLACE, applicationId=application_1452791807475_0001, dagName=[A27728F4E5C54B3FA02BAE87E6422171/D9768DF31E694AC681A71B3FC6F8AACF] every/(1/1) ...tform/ALL_UNKNOWN_REPLACE
2016-01-14 17:16:47,634 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:47,634 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791807475_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:47,634 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: CCD16C333F334EE08B36D4E1E78E04A3 : {3B4CFACAFD3241EDABFAE2DAD7EB2DBB={InputName=3B4CFACAFD3241EDABFAE2DAD7EB2DBB}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:47,634 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 3B4CFACAFD3241EDABFAE2DAD7EB2DBB, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:47,635 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791807475_0001_1_00 [CCD16C333F334EE08B36D4E1E78E04A3]
2016-01-14 17:16:47,635 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791807475_0001_1_00 [CCD16C333F334EE08B36D4E1E78E04A3]
2016-01-14 17:16:47,635 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791807475_0001_1_00 [CCD16C333F334EE08B36D4E1E78E04A3]
2016-01-14 17:16:47,635 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791807475_0001_1_00 [CCD16C333F334EE08B36D4E1E78E04A3]. Starting root input initializers: 1
2016-01-14 17:16:47,635 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791807475_0001_1_00 [CCD16C333F334EE08B36D4E1E78E04A3] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:47,636 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791807475_0001_1_01 [B27AFB27853E4D72B3907EB9F4A97A56]
2016-01-14 17:16:47,638 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:47,638 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791807475_0001_1_01 [B27AFB27853E4D72B3907EB9F4A97A56]
2016-01-14 17:16:47,638 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791807475_0001_1_01 [B27AFB27853E4D72B3907EB9F4A97A56]
2016-01-14 17:16:47,638 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791807475_0001_1_01 [B27AFB27853E4D72B3907EB9F4A97A56]
2016-01-14 17:16:47,638 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=41E495B845094A3C8FDA43E9CBD8DE17, vertexId=vertex_1452791807475_0001_1_01 [B27AFB27853E4D72B3907EB9F4A97A56], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:47,638 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=41E495B845094A3C8FDA43E9CBD8DE17, vertexId=vertex_1452791807475_0001_1_01 [B27AFB27853E4D72B3907EB9F4A97A56]
2016-01-14 17:16:47,640 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=41E495B845094A3C8FDA43E9CBD8DE17, vertexName=B27AFB27853E4D72B3907EB9F4A97A56, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:47,640 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=41E495B845094A3C8FDA43E9CBD8DE17, vertexId=vertex_1452791807475_0001_1_01 [B27AFB27853E4D72B3907EB9F4A97A56]
2016-01-14 17:16:47,641 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 3B4CFACAFD3241EDABFAE2DAD7EB2DBB on vertex vertex_1452791807475_0001_1_00 [CCD16C333F334EE08B36D4E1E78E04A3]
2016-01-14 17:16:47,641 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791807475_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B27AFB27853E4D72B3907EB9F4A97A56, vertexId=vertex_1452791807475_0001_1_01, initRequestedTime=1452791807636, initedTime=1452791807641, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:47,641 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:47,642 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791807475_0001_1_01 [B27AFB27853E4D72B3907EB9F4A97A56] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:47,643 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:47,643 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 3B4CFACAFD3241EDABFAE2DAD7EB2DBB asking for 1 tasks. Headroom: 686 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:47,644 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:47,644 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:47,645 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:47,645 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 3B4CFACAFD3241EDABFAE2DAD7EB2DBB on vertex vertex_1452791807475_0001_1_00 [CCD16C333F334EE08B36D4E1E78E04A3]
2016-01-14 17:16:47,645 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791807475_0001_1_00 [CCD16C333F334EE08B36D4E1E78E04A3] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:47,646 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {3B4CFACAFD3241EDABFAE2DAD7EB2DBB=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:47,646 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791807475_0001_1_00 [CCD16C333F334EE08B36D4E1E78E04A3] parallelism set to 1
2016-01-14 17:16:47,646 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791807475_0001_1_00 [CCD16C333F334EE08B36D4E1E78E04A3] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:47,646 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791807475_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791807475_0001_1_00, eventCount=1
2016-01-14 17:16:47,647 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791807475_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CCD16C333F334EE08B36D4E1E78E04A3, vertexId=vertex_1452791807475_0001_1_00, initRequestedTime=1452791807634, initedTime=1452791807647, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:47,647 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791807475_0001_1_00 [CCD16C333F334EE08B36D4E1E78E04A3] with distanceFromRoot: 0
2016-01-14 17:16:47,647 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:47,647 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791807475_0001_1_00 [CCD16C333F334EE08B36D4E1E78E04A3] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:47,647 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in CCD16C333F334EE08B36D4E1E78E04A3
2016-01-14 17:16:47,647 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791807475_0001_1_00 [CCD16C333F334EE08B36D4E1E78E04A3]
2016-01-14 17:16:47,647 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:47,648 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791807475_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791807475_0001_1_00, startRequestedTime=1452791807647, startedTime=1452791807647
2016-01-14 17:16:47,648 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791807475_0001_1_00 [CCD16C333F334EE08B36D4E1E78E04A3] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:47,648 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:47,648 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791807475_0001_1][Event:TASK_STARTED]: vertexName=CCD16C333F334EE08B36D4E1E78E04A3, taskId=task_1452791807475_0001_1_00_000000, scheduledTime=1452791807648, launchTime=1452791807648
2016-01-14 17:16:47,648 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791807475_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:47,648 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791807475_0001_1_00 for vertex: vertex_1452791807475_0001_1_01 [B27AFB27853E4D72B3907EB9F4A97A56] numStartedSources: 1 numSources: 1
2016-01-14 17:16:47,649 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791807475_0001_1_01 [B27AFB27853E4D72B3907EB9F4A97A56] with distanceFromRoot: 1
2016-01-14 17:16:47,649 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791807475_0001_1_00 [CCD16C333F334EE08B36D4E1E78E04A3]
2016-01-14 17:16:47,649 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: CCD16C333F334EE08B36D4E1E78E04A3 in vertex: B27AFB27853E4D72B3907EB9F4A97A56
2016-01-14 17:16:47,649 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: B27AFB27853E4D72B3907EB9F4A97A56 with 1 source tasks and 1 pending tasks
2016-01-14 17:16:47,650 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:47,650 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791807475_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791807475_0001_1_01, startRequestedTime=1452791807649, startedTime=1452791807649
2016-01-14 17:16:47,650 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791807475_0001_1_01 [B27AFB27853E4D72B3907EB9F4A97A56] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:47,650 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [A27728F4E5C54B3FA02BAE87E6422171/D9768DF31E694AC681A71B3FC6F8AACF] every/(1/1) ...tform/ALL_UNKNOWN_REPLACE, VertexName: CCD16C333F334EE08B36D4E1E78E04A3, VertexParallelism: 1, TaskAttemptID:attempt_1452791807475_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=3B4CFACAFD3241EDABFAE2DAD7EB2DBB, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B27AFB27853E4D72B3907EB9F4A97A56, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:47,650 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791807475_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:47,651 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:47,651 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:47,652 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791807475_0001, with dag id: dag_1452791807475_0001_1
2016-01-14 17:16:47,652 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:47,652 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:47,652 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791807475_0001_00_000001
2016-01-14 17:16:47,652 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791807475_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:47,653 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791807475_0001_00_000001
2016-01-14 17:16:47,654 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:47,654 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791807475_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:47,654 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:47,654 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791807475_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791807475_0001_00_000001, launchTime=1452791807654
2016-01-14 17:16:47,655 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791807475_0001_00_000001
2016-01-14 17:16:47,656 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791807475_0001_1_00_000000_0] to container: [container_1452791807475_0001_00_000001]
2016-01-14 17:16:47,656 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791807475_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:47,657 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791807475_0001_1_00_000000_0] started. Is using containerId: [container_1452791807475_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:47,657 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791807475_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CCD16C333F334EE08B36D4E1E78E04A3, taskAttemptId=attempt_1452791807475_0001_1_00_000000_0, startTime=1452791807656, containerId=container_1452791807475_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791807475_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:47,657 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791807475_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:47,657 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791807475_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:47,657 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791807475_0001_00_000001 given task: attempt_1452791807475_0001_1_00_000000_0
2016-01-14 17:16:47,657 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791807475_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791807475_0001_1_00_000000_0
2016-01-14 17:16:47,657 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:47 Starting to run new task attempt: attempt_1452791807475_0001_1_00_000000_0
2016-01-14 17:16:47,658 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791807475_0001_1_00_000000_0
2016-01-14 17:16:47,658 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:47,658 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:47,658 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:47,658 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:47,658 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:47,658 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:47,658 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [A27728F4E5C54B3FA02BAE87E6422171/D9768DF31E694AC681A71B3FC6F8AACF] every/(1/1) ...tform/ALL_UNKNOWN_REPLACE, VertexName: CCD16C333F334EE08B36D4E1E78E04A3, VertexParallelism: 1, TaskAttemptID:attempt_1452791807475_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=3B4CFACAFD3241EDABFAE2DAD7EB2DBB, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B27AFB27853E4D72B3907EB9F4A97A56, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:47,658 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=719847424
2016-01-14 17:16:47,659 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791807475_0001_1_00_000000_0
2016-01-14 17:16:47,659 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:47,659 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:47,660 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:47,660 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=3B4CFACAFD3241EDABFAE2DAD7EB2DBB, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:47,660 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:47,661 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 3B4CFACAFD3241EDABFAE2DAD7EB2DBB
2016-01-14 17:16:47,661 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 3B4CFACAFD3241EDABFAE2DAD7EB2DBB
2016-01-14 17:16:47,673 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=B27AFB27853E4D72B3907EB9F4A97A56, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:47,674 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:47,674 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: B27AFB27853E4D72B3907EB9F4A97A56
2016-01-14 17:16:47,675 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:47,675 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 3B4CFACAFD3241EDABFAE2DAD7EB2DBB
2016-01-14 17:16:47,675 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 3B4CFACAFD3241EDABFAE2DAD7EB2DBB
2016-01-14 17:16:47,675 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:47,675 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: B27AFB27853E4D72B3907EB9F4A97A56
2016-01-14 17:16:47,676 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: B27AFB27853E4D72B3907EB9F4A97A56
2016-01-14 17:16:47,677 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791807475_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:47,678 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:47,681 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CCD16C333F334EE08B36D4E1E78E04A3, ordinal: 0
2016-01-14 17:16:47,682 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CCD16C333F334EE08B36D4E1E78E04A3, mem on start (mb), free: 430, total: 687, max: 687
2016-01-14 17:16:47,682 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:47,682 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:47,682 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:47,682 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:47,683 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:47,683 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:47,683 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:47,683 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:47,684 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:47,684 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 720371712, TotalAvailable: 482297774, TotalRequested/TotalJVMHeap:0.15
2016-01-14 17:16:47,684 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 3B4CFACAFD3241EDABFAE2DAD7EB2DBB, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:47,684 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, B27AFB27853E4D72B3907EB9F4A97A56, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:47,685 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:47,685 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 3B4CFACAFD3241EDABFAE2DAD7EB2DBB being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:47,685 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 3B4CFACAFD3241EDABFAE2DAD7EB2DBB
2016-01-14 17:16:47,685 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:47,685 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 3B4CFACAFD3241EDABFAE2DAD7EB2DBB
2016-01-14 17:16:47,685 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:47,685 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:47,686 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:47,686 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791807475_0001_1_00_000000_0
2016-01-14 17:16:47,686 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:47,686 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@7b09519b
2016-01-14 17:16:47,687 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@7b09519b
2016-01-14 17:16:47,687 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:47,687 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:47,690 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:47,690 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 3B4CFACAFD3241EDABFAE2DAD7EB2DBB
2016-01-14 17:16:47,690 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: D6D4ABD020744A8F9C34A15085036C4B
2016-01-14 17:16:47,690 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] D6D4ABD020744A8F9C34A15085036C4B
2016-01-14 17:16:47,691 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@737dc58c; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@4634a590; comparator=cascading.tuple.hadoop.util.TupleComparator@2a61b4f2; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:47,691 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:47,691 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:47,692 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:47,705 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:47,705 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:47,705 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:47,706 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:47,706 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:47,706 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:47,706 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:47,707 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CCD16C333F334EE08B36D4E1E78E04A3, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:47,709 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791807475_0001_1_00_000000_0
2016-01-14 17:16:47,710 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CCD16C333F334EE08B36D4E1E78E04A3, mem on close (mb), free: 329, total: 687, max: 687
2016-01-14 17:16:47,710 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:47,710 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:47,710 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:47,710 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:47,713 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:47 Completed running task attempt: attempt_1452791807475_0001_1_00_000000_0
2016-01-14 17:16:47,718 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791807475_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:47,718 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791807475_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791807475_0001_1_00, eventCount=1
2016-01-14 17:16:47,719 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=11, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=720371712, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791807707, Process_Duration=2, Process_End_Time=1452791807709, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:47,719 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:47,719 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791807475_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CCD16C333F334EE08B36D4E1E78E04A3, taskAttemptId=attempt_1452791807475_0001_1_00_000000_0, startTime=1452791807656, finishTime=1452791807719, timeTaken=63, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=11, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=720371712, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791807707, Process_Duration=2, Process_End_Time=1452791807709, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:47,720 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791807475_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:47,721 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791807475_0001_1_00_000000_0
2016-01-14 17:16:47,721 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:47,721 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:47,721 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791807475_0001_1][Event:TASK_FINISHED]: vertexName=CCD16C333F334EE08B36D4E1E78E04A3, taskId=task_1452791807475_0001_1_00_000000, startTime=1452791807656, finishTime=1452791807721, timeTaken=65, status=SUCCEEDED, successfulAttemptID=attempt_1452791807475_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=11, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=720371712, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791807707, Process_Duration=2, Process_End_Time=1452791807709, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:47,721 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791807475_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:47,721 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791807475_0001_1_00 [CCD16C333F334EE08B36D4E1E78E04A3] : 1
2016-01-14 17:16:47,721 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791807475_0001_1_00 [CCD16C333F334EE08B36D4E1E78E04A3]
2016-01-14 17:16:47,722 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791807475_0001_1][Event:VERTEX_FINISHED]: vertexName=CCD16C333F334EE08B36D4E1E78E04A3, vertexId=vertex_1452791807475_0001_1_00, initRequestedTime=1452791807634, initedTime=1452791807647, startRequestedTime=1452791807647, startedTime=1452791807647, finishTime=1452791807721, timeTaken=74, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=11, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=720371712, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791807707, Process_Duration=2, Process_End_Time=1452791807709, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791807656, firstTasksToStart=[ task_1452791807475_0001_1_00_000000 ], lastTaskFinishTime=1452791807719, lastTasksToFinish=[ task_1452791807475_0001_1_00_000000 ], minTaskDuration=63, maxTaskDuration=63, avgTaskDuration=63.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791807475_0001_1_00_000000 ], longestDurationTasks=[ task_1452791807475_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:47,722 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791807475_0001_1_00 [CCD16C333F334EE08B36D4E1E78E04A3] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:47,722 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791807475_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:47,722 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791807475_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791807475_0001_00_000001, stoppedTime=1452791807722, exitStatus=0
2016-01-14 17:16:47,723 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:47,723 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:47,723 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:47,723 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:47,723 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791807475_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:47,723 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791807475_0001_1_01 [B27AFB27853E4D72B3907EB9F4A97A56] attempt: attempt_1452791807475_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:47,724 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: B27AFB27853E4D72B3907EB9F4A97A56
2016-01-14 17:16:47,724 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791807475_0001_1_01 [B27AFB27853E4D72B3907EB9F4A97A56]
2016-01-14 17:16:47,724 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791807475_0001_1_00 [CCD16C333F334EE08B36D4E1E78E04A3] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:47,724 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791807475_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:47,724 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791807475_0001_1][Event:TASK_STARTED]: vertexName=B27AFB27853E4D72B3907EB9F4A97A56, taskId=task_1452791807475_0001_1_01_000000, scheduledTime=1452791807724, launchTime=1452791807724
2016-01-14 17:16:47,724 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791807475_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:47,724 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791807475_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:47,725 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791807475_0001_1_01 [B27AFB27853E4D72B3907EB9F4A97A56]
2016-01-14 17:16:47,725 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [A27728F4E5C54B3FA02BAE87E6422171/D9768DF31E694AC681A71B3FC6F8AACF] every/(1/1) ...tform/ALL_UNKNOWN_REPLACE, VertexName: B27AFB27853E4D72B3907EB9F4A97A56, VertexParallelism: 1, TaskAttemptID:attempt_1452791807475_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=CCD16C333F334EE08B36D4E1E78E04A3, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=41E495B845094A3C8FDA43E9CBD8DE17, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:47,725 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791807475_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:47,725 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:47,723 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791807475_0001_00_000001
2016-01-14 17:16:47,725 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:47,726 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791807475_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:47,726 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791807475_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:47,726 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791807475_0001_00_000001, Exiting
2016-01-14 17:16:47,726 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791807475_0001_00_000001
2016-01-14 17:16:47,726 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791807475_0001_00_000001
2016-01-14 17:16:47,727 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791807475_0001_00_000002
2016-01-14 17:16:47,727 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791807475_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:47,728 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791807475_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:47,728 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791807475_0001_00_000001 completed successfully
2016-01-14 17:16:47,729 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791807475_0001_00_000002
2016-01-14 17:16:47,729 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791807475_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:47,730 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 17:16:47,732 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:47,732 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791807475_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791807475_0001_00_000002, launchTime=1452791807732
2016-01-14 17:16:47,732 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791807475_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:47,733 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791807475_0001_00_000002
2016-01-14 17:16:47,733 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791807475_0001_1_01_000000_0] to container: [container_1452791807475_0001_00_000002]
2016-01-14 17:16:47,733 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791807475_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:47,733 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791807475_0001_00_000002 given task: attempt_1452791807475_0001_1_01_000000_0
2016-01-14 17:16:47,734 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791807475_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791807475_0001_1_01_000000_0
2016-01-14 17:16:47,734 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791807475_0001_1_01_000000_0] started. Is using containerId: [container_1452791807475_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:47 Starting to run new task attempt: attempt_1452791807475_0001_1_01_000000_0
2016-01-14 17:16:47,734 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791807475_0001_1_01_000000_0
2016-01-14 17:16:47,734 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:47,734 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:47,735 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:47,735 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:47,735 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791807475_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B27AFB27853E4D72B3907EB9F4A97A56, taskAttemptId=attempt_1452791807475_0001_1_01_000000_0, startTime=1452791807734, containerId=container_1452791807475_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791807475_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:47,735 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:47,735 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:47,735 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791807475_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:47,735 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:47,736 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [A27728F4E5C54B3FA02BAE87E6422171/D9768DF31E694AC681A71B3FC6F8AACF] every/(1/1) ...tform/ALL_UNKNOWN_REPLACE, VertexName: B27AFB27853E4D72B3907EB9F4A97A56, VertexParallelism: 1, TaskAttemptID:attempt_1452791807475_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=CCD16C333F334EE08B36D4E1E78E04A3, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=41E495B845094A3C8FDA43E9CBD8DE17, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:47,735 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791807475_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:47,736 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=720371712
2016-01-14 17:16:47,736 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791807475_0001_1_01_000000_0
2016-01-14 17:16:47,736 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:47,737 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:47,737 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791807475_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:47,737 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:47,737 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=CCD16C333F334EE08B36D4E1E78E04A3, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:47,737 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=41E495B845094A3C8FDA43E9CBD8DE17, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:47,738 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:47,738 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 41E495B845094A3C8FDA43E9CBD8DE17
2016-01-14 17:16:47,738 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:47,739 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 41E495B845094A3C8FDA43E9CBD8DE17
2016-01-14 17:16:47,738 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:47,739 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: CCD16C333F334EE08B36D4E1E78E04A3
2016-01-14 17:16:47,741 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:47,742 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 648334528, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:47,742 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:47,743 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 648334528, type: INPUT, componentVertexName: CCD16C333F334EE08B36D4E1E78E04A3
2016-01-14 17:16:47,743 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: CCD16C333F334EE08B36D4E1E78E04A3
2016-01-14 17:16:47,745 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B27AFB27853E4D72B3907EB9F4A97A56, ordinal: 1
2016-01-14 17:16:47,745 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B27AFB27853E4D72B3907EB9F4A97A56, mem on start (mb), free: 323, total: 687, max: 687
2016-01-14 17:16:47,745 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:47,746 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:47,746 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:47,748 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:47,748 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 41E495B845094A3C8FDA43E9CBD8DE17
2016-01-14 17:16:47,748 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:47,748 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:47,749 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:47,749 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:47,749 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:47,749 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:47,749 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 648334528, TotalRequestedScaled: 5.984626412307693E8, TotalJVMHeap: 720371712, TotalAvailable: 482649047, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:47,750 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 41E495B845094A3C8FDA43E9CBD8DE17, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:47,750 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, CCD16C333F334EE08B36D4E1E78E04A3, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=648334528, allocated=482649047
2016-01-14 17:16:47,750 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:47,750 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: CCD16C333F334EE08B36D4E1E78E04A3 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:47,750 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: CCD16C333F334EE08B36D4E1E78E04A3
2016-01-14 17:16:47,751 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:47,751 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:47,751 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:47,751 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:47,751 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: CCD16C333F334EE08B36D4E1E78E04A3 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:47,751 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=648334528, postMergeMem=0, RuntimeTotalAvailable=482649047. Updated to: ShuffleMem=482649047, postMergeMem: 0
2016-01-14 17:16:47,752 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=482649047, maxSingleShuffleLimit=120662264, mergeThreshold=434384128, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:47,752 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:47,752 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: CCD16C333F334EE08B36D4E1E78E04A3
2016-01-14 17:16:47,752 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:47,753 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791807475_0001_1_01_000000_0
2016-01-14 17:16:47,753 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791807475_0001_1_00_000000_0_10279, runDuration: 42436, ]
2016-01-14 17:16:47,754 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791807475_0001_1_00_000000_0_10279] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791807754, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 17:16:47,754 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : CCD16C333F334EE08B36D4E1E78E04A3
2016-01-14 17:16:47,754 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:47,754 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [CCD16C333F334EE08B36D4E1E78E04A3] #47 in 1ms
2016-01-14 17:16:47,754 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [CCD16C333F334EE08B36D4E1E78E04A3] #47, status:false, isInterrupted:false
2016-01-14 17:16:47,754 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:47,755 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:47,755 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:47,755 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:47,755 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:47,755 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : CCD16C333F334EE08B36D4E1E78E04A3
2016-01-14 17:16:47,755 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:47,755 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:47,756 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: D6D4ABD020744A8F9C34A15085036C4B
2016-01-14 17:16:47,756 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_REPLACE"], id: 41E495B845094A3C8FDA43E9CBD8DE17
2016-01-14 17:16:47,756 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_REPLACE"]
2016-01-14 17:16:47,756 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] D6D4ABD020744A8F9C34A15085036C4B, for 1 inputs
2016-01-14 17:16:47,756 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B27AFB27853E4D72B3907EB9F4A97A56, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:47,757 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@71d7803b; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@7b23dbe0
2016-01-14 17:16:47,757 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:47,758 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(92)) - caught throwable
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:47,758 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:47,758 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:47,758 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791807475_0001_1_01_000000_0
2016-01-14 17:16:47,759 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791807475_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:47,759 INFO  task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1452791807475_0001_1_01_000000_0
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:47,762 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791807475_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B27AFB27853E4D72B3907EB9F4A97A56, taskAttemptId=attempt_1452791807475_0001_1_01_000000_0, startTime=1452791807734, finishTime=1452791807761, timeTaken=27, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=720371712, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=3, MERGE_PHASE_TIME=4, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791807757, Process_Duration=2, Process_End_Time=1452791807759, Read_Duration=0, Tuples_Read=1
2016-01-14 17:16:47,762 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791807475_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-01-14 17:16:47,762 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:47,762 INFO  impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1452791807475_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-01-14 17:16:47,762 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791807475_0001_1][Event:TASK_FINISHED]: vertexName=B27AFB27853E4D72B3907EB9F4A97A56, taskId=task_1452791807475_0001_1_01_000000, startTime=1452791807734, finishTime=1452791807762, timeTaken=28, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
], counters=Counters: 0
2016-01-14 17:16:47,762 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791807475_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-01-14 17:16:47,762 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:47,762 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:47,762 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791807475_0001_1_01 [B27AFB27853E4D72B3907EB9F4A97A56] : 1
2016-01-14 17:16:47,763 INFO  impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1452791807475_0001_1_01 [B27AFB27853E4D72B3907EB9F4A97A56] because task failed: task_1452791807475_0001_1_01_000000
2016-01-14 17:16:47,763 INFO  impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1452791807475_0001_1_01 [B27AFB27853E4D72B3907EB9F4A97A56] due to trigger: OWN_TASK_FAILURE
2016-01-14 17:16:47,763 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-01-14 17:16:47,763 INFO  impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1452791807475_0001_1_01 [B27AFB27853E4D72B3907EB9F4A97A56]
2016-01-14 17:16:47,763 INFO  impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=41E495B845094A3C8FDA43E9CBD8DE17, vertexId=vertex_1452791807475_0001_1_01 [B27AFB27853E4D72B3907EB9F4A97A56]
2016-01-14 17:16:47,764 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791807475_0001_1][Event:VERTEX_FINISHED]: vertexName=B27AFB27853E4D72B3907EB9F4A97A56, vertexId=vertex_1452791807475_0001_1_01, initRequestedTime=1452791807636, initedTime=1452791807641, startRequestedTime=1452791807649, startedTime=1452791807649, finishTime=1452791807764, timeTaken=115, status=FAILED, diagnostics=Task failed, taskId=task_1452791807475_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1452791807475_0001_1_01 [B27AFB27853E4D72B3907EB9F4A97A56] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1452791807734, firstTasksToStart=[ task_1452791807475_0001_1_01_000000 ], lastTaskFinishTime=1452791807761, lastTasksToFinish=[ task_1452791807475_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[  ], longestDurationTasks=[  ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-01-14 17:16:47,764 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791807475_0001_1_01 [B27AFB27853E4D72B3907EB9F4A97A56] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-01-14 17:16:47,764 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:47,765 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791807475_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791807475_0001_00_000002, stoppedTime=1452791807765, exitStatus=0
2016-01-14 17:16:47,765 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791807475_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:47,765 INFO  node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1452791807475_0001_1_01_000000_0 failed: true container: container_1452791807475_0001_00_000002 numFailedTAs: 0
2016-01-14 17:16:47,765 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791807475_0001_1_01 [B27AFB27853E4D72B3907EB9F4A97A56] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-01-14 17:16:47,765 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:47,765 INFO  impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-01-14 17:16:47,765 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791807475_0001_1
2016-01-14 17:16:47,765 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1452791807475_0001_1_00 [CCD16C333F334EE08B36D4E1E78E04A3]
2016-01-14 17:16:47,765 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 41E495B845094A3C8FDA43E9CBD8DE17 for vertex: vertex_1452791807475_0001_1_01 [B27AFB27853E4D72B3907EB9F4A97A56]
2016-01-14 17:16:47,766 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791807475_0001_1, queueSize=1
2016-01-14 17:16:47,767 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:47,767 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791807475_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791807475_0001_1, startTime=1452791807634, finishTime=1452791807766, timeTaken=132, status=FAILED, diagnostics=Vertex failed, vertexName=B27AFB27853E4D72B3907EB9F4A97A56, vertexId=vertex_1452791807475_0001_1_01, diagnostics=[Task failed, taskId=task_1452791807475_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1452791807475_0001_1_01 [B27AFB27853E4D72B3907EB9F4A97A56] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=11, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=720371712, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791807707, Process_Duration=2, Process_End_Time=1452791807709, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:47,767 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791807475_0001_1 finished with state: FAILED
2016-01-14 17:16:47,767 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791807475_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:47,767 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791807475_0001_00_000002. Relying on regular task shutdown for it to end
2016-01-14 17:16:47 Completed Dag: dag_1452791807475_0001_1
2016-01-14 17:16:47,767 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791807475_0001_1, dagState=FAILED
2016-01-14 17:16:47,767 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791807475_0001_1_post
2016-01-14 17:16:47,767 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:47,767 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:47,768 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791807475_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:47,768 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=720371712, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=3, MERGE_PHASE_TIME=4, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791807757, Process_Duration=2, Process_End_Time=1452791807759, Read_Duration=0, Tuples_Read=1]]
2016-01-14 17:16:47,768 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B27AFB27853E4D72B3907EB9F4A97A56, mem on close (mb), free: 319, total: 687, max: 687
2016-01-14 17:16:47,768 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=B27AFB27853E4D72B3907EB9F4A97A56, index=1
2016-01-14 17:16:47,768 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: CCD16C333F334EE08B36D4E1E78E04A3
2016-01-14 17:16:47,768 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:47,768 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:47,769 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:47,769 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:47,769 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:47,769 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=B27AFB27853E4D72B3907EB9F4A97A56, sourceVertex=CCD16C333F334EE08B36D4E1E78E04A3
2016-01-14 17:16:47,769 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=B27AFB27853E4D72B3907EB9F4A97A56, sourceVertex=41E495B845094A3C8FDA43E9CBD8DE17
2016-01-14 17:16:47,769 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791807475_0001_00_000002
2016-01-14 17:16:47,769 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791807475_0001_00_000002
2016-01-14 17:16:47,770 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791807475_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-01-14 17:16:47,770 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1452791807475_0001_00_000002 completed but with errors
2016-01-14 17:16:47,770 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1452791807475_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-01-14 17:16:47,770 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791807475_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:47,770 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791807475_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-01-14 17:16:47,779 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...tform/ALL_UNKNOWN_REPLACE, applicationId=application_1452791807475_0001
2016-01-14 17:16:47,781 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:47,781 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:47,781 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:47,781 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:47,781 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:47,782 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:47,782 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:47,782 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:47,782 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:47,782 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:47,782 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:47,782 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:47,783 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:47,785 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A27728F4E5C54B3FA02BAE87E6422171/D9768DF31E694AC681A71B3FC6F8AACF/.tez/application_1452791807475_0001
2016-01-14 17:16:47,786 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping all jobs
2016-01-14 17:16:47,786 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping: (1/1) ...tform/ALL_UNKNOWN_REPLACE
2016-01-14 17:16:47,787 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopped all jobs
2016-01-14 17:16:47,787 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.330
2016-01-14 17:16:47,787 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_UNKNOWN_REPLACE
2016-01-14 17:16:47,807 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:47,808 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:47,837 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.008
2016-01-14 17:16:47,839 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-01-14 17:16:47,839 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:47,839 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:47,839 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:47,852 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:47,856 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_SWAP
2016-01-14 17:16:47,895 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:47,896 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:47,896 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_SWAP"]
2016-01-14 17:16:47,896 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:47,896 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:47,896 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:47,898 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...platform/ALL_UNKNOWN_SWAP
2016-01-14 17:16:47,898 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:47,908 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:47,913 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:47,913 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:47,913 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:47,922 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/BF7A219A04154B7AB49FC3B3EC9ACD7A/AD016E9AFC194B9EB0836320381C40C0/.tez/application_1452791807913_0001 doesn't exist and is created
2016-01-14 17:16:47,937 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:47,937 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:47,938 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/BF7A219A04154B7AB49FC3B3EC9ACD7A/AD016E9AFC194B9EB0836320381C40C0/.tez/application_1452791807913_0001
2016-01-14 17:16:47,940 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791807913_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:47,950 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:47,951 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:47,951 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:47,952 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:47,952 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/BF7A219A04154B7AB49FC3B3EC9ACD7A/AD016E9AFC194B9EB0836320381C40C0/.tez/application_1452791807913_0001/localmode-log-dir
2016-01-14 17:16:47,952 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/BF7A219A04154B7AB49FC3B3EC9ACD7A/AD016E9AFC194B9EB0836320381C40C0/.tez/application_1452791807913_0001/localmode-log-dir/history.txt.appattempt_1452791807913_0001_000000, maxErrors=10
2016-01-14 17:16:47,952 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:47,952 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791807913_0001_000000, appSubmitTime=1452791807940, launchTime=1452791807940
2016-01-14 17:16:47,954 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:47,955 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:47,955 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:47,958 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:47,958 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:50411
2016-01-14 17:16:47,959 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:47,961 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791807913_0001_000000, startTime=1452791807961
2016-01-14 17:16:47,961 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:47,961 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:48,037 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:48,038 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:48,038 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...platform/ALL_UNKNOWN_SWAP, applicationId=application_1452791807913_0001, dagName=[BF7A219A04154B7AB49FC3B3EC9ACD7A/AD016E9AFC194B9EB0836320381C40C0] every/(1/1) ...platform/ALL_UNKNOWN_SWAP
2016-01-14 17:16:48,051 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [BF7A219A04154B7AB49FC3B3EC9ACD7A/AD016E9AFC194B9EB0836320381C40C0] every/(1/1) ...platform/ALL_UNKNOWN_SWAP
2016-01-14 17:16:48,053 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791807913_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/BF7A219A04154B7AB49FC3B3EC9ACD7A/AD016E9AFC194B9EB0836320381C40C0/.tez/application_1452791807913_0001/localmode-log-dir/dag_1452791807913_0001_1.dot
2016-01-14 17:16:48,053 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/BF7A219A04154B7AB49FC3B3EC9ACD7A/AD016E9AFC194B9EB0836320381C40C0/.tez/application_1452791807913_0001/localmode-log-dir/dag_1452791807913_0001_1-tez-dag.pb.txt
2016-01-14 17:16:48,059 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791807913_0001_1
2016-01-14 17:16:48,060 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:48,060 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [BF7A219A04154B7AB49FC3B3EC9ACD7A/AD016E9AFC194B9EB0836320381C40C0] every/(1/1) ...platform/ALL_UNKNOWN_SWAP
2016-01-14 17:16:48 Running Dag: dag_1452791807913_0001_1
2016-01-14 17:16:48,063 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 17:16:48,063 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 17:16:48,073 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:48,073 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791807913_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791807913_0001_1, submitTime=1452791808052
2016-01-14 17:16:48,074 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex CB83CB6CCE634AB68F06CC68C0E1D975
2016-01-14 17:16:48,074 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:48,075 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:48,075 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791807913_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791807913_0001_1, initTime=1452791808073
2016-01-14 17:16:48,075 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791807913_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:48,075 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791807913_0001_1][Event:DAG_STARTED]: dagID=dag_1452791807913_0001_1, startTime=1452791808075
2016-01-14 17:16:48,075 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:48,075 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791807913_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:48,075 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: AC80E9F8F4F84A9E8F9E48C8EBF6F597 : {757CC289630D4EE6A47A3645029D7DEA={InputName=757CC289630D4EE6A47A3645029D7DEA}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:48,075 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 757CC289630D4EE6A47A3645029D7DEA, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:48,075 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...platform/ALL_UNKNOWN_SWAP, applicationId=application_1452791807913_0001, dagName=[BF7A219A04154B7AB49FC3B3EC9ACD7A/AD016E9AFC194B9EB0836320381C40C0] every/(1/1) ...platform/ALL_UNKNOWN_SWAP
2016-01-14 17:16:48,076 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:48,075 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791807913_0001_1_00 [AC80E9F8F4F84A9E8F9E48C8EBF6F597]
2016-01-14 17:16:48,076 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791807913_0001_1_00 [AC80E9F8F4F84A9E8F9E48C8EBF6F597]
2016-01-14 17:16:48,076 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791807913_0001_1_00 [AC80E9F8F4F84A9E8F9E48C8EBF6F597]
2016-01-14 17:16:48,076 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791807913_0001_1_00 [AC80E9F8F4F84A9E8F9E48C8EBF6F597]. Starting root input initializers: 1
2016-01-14 17:16:48,077 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791807913_0001_1_00 [AC80E9F8F4F84A9E8F9E48C8EBF6F597] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:48,077 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791807913_0001_1_01 [CB83CB6CCE634AB68F06CC68C0E1D975]
2016-01-14 17:16:48,077 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 757CC289630D4EE6A47A3645029D7DEA on vertex vertex_1452791807913_0001_1_00 [AC80E9F8F4F84A9E8F9E48C8EBF6F597]
2016-01-14 17:16:48,080 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:48,080 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:48,080 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791807913_0001_1_01 [CB83CB6CCE634AB68F06CC68C0E1D975]
2016-01-14 17:16:48,081 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791807913_0001_1_01 [CB83CB6CCE634AB68F06CC68C0E1D975]
2016-01-14 17:16:48,081 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791807913_0001_1_01 [CB83CB6CCE634AB68F06CC68C0E1D975]
2016-01-14 17:16:48,081 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=D360597DC62F4DA887D53D81F871AF68, vertexId=vertex_1452791807913_0001_1_01 [CB83CB6CCE634AB68F06CC68C0E1D975], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:48,081 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=D360597DC62F4DA887D53D81F871AF68, vertexId=vertex_1452791807913_0001_1_01 [CB83CB6CCE634AB68F06CC68C0E1D975]
2016-01-14 17:16:48,081 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 757CC289630D4EE6A47A3645029D7DEA asking for 1 tasks. Headroom: 687 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:48,082 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:48,082 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:48,083 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=D360597DC62F4DA887D53D81F871AF68, vertexName=CB83CB6CCE634AB68F06CC68C0E1D975, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:48,083 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=D360597DC62F4DA887D53D81F871AF68, vertexId=vertex_1452791807913_0001_1_01 [CB83CB6CCE634AB68F06CC68C0E1D975]
2016-01-14 17:16:48,084 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:48,084 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:48,085 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791807913_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CB83CB6CCE634AB68F06CC68C0E1D975, vertexId=vertex_1452791807913_0001_1_01, initRequestedTime=1452791808077, initedTime=1452791808084, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:48,085 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791807913_0001_1_01 [CB83CB6CCE634AB68F06CC68C0E1D975] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:48,085 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 757CC289630D4EE6A47A3645029D7DEA on vertex vertex_1452791807913_0001_1_00 [AC80E9F8F4F84A9E8F9E48C8EBF6F597]
2016-01-14 17:16:48,085 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791807913_0001_1_00 [AC80E9F8F4F84A9E8F9E48C8EBF6F597] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:48,086 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {757CC289630D4EE6A47A3645029D7DEA=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:48,086 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791807913_0001_1_00 [AC80E9F8F4F84A9E8F9E48C8EBF6F597] parallelism set to 1
2016-01-14 17:16:48,086 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791807913_0001_1_00 [AC80E9F8F4F84A9E8F9E48C8EBF6F597] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:48,086 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791807913_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791807913_0001_1_00, eventCount=1
2016-01-14 17:16:48,087 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:48,087 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791807913_0001_1][Event:VERTEX_INITIALIZED]: vertexName=AC80E9F8F4F84A9E8F9E48C8EBF6F597, vertexId=vertex_1452791807913_0001_1_00, initRequestedTime=1452791808075, initedTime=1452791808087, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:48,087 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791807913_0001_1_00 [AC80E9F8F4F84A9E8F9E48C8EBF6F597] with distanceFromRoot: 0
2016-01-14 17:16:48,087 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791807913_0001_1_00 [AC80E9F8F4F84A9E8F9E48C8EBF6F597] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:48,088 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in AC80E9F8F4F84A9E8F9E48C8EBF6F597
2016-01-14 17:16:48,088 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791807913_0001_1_00 [AC80E9F8F4F84A9E8F9E48C8EBF6F597]
2016-01-14 17:16:48,088 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:48,088 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791807913_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791807913_0001_1_00, startRequestedTime=1452791808088, startedTime=1452791808088
2016-01-14 17:16:48,089 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791807913_0001_1_00 [AC80E9F8F4F84A9E8F9E48C8EBF6F597] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:48,089 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:48,089 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791807913_0001_1][Event:TASK_STARTED]: vertexName=AC80E9F8F4F84A9E8F9E48C8EBF6F597, taskId=task_1452791807913_0001_1_00_000000, scheduledTime=1452791808089, launchTime=1452791808089
2016-01-14 17:16:48,090 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791807913_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:48,090 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791807913_0001_1_00 for vertex: vertex_1452791807913_0001_1_01 [CB83CB6CCE634AB68F06CC68C0E1D975] numStartedSources: 1 numSources: 1
2016-01-14 17:16:48,090 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791807913_0001_1_01 [CB83CB6CCE634AB68F06CC68C0E1D975] with distanceFromRoot: 1
2016-01-14 17:16:48,090 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791807913_0001_1_00 [AC80E9F8F4F84A9E8F9E48C8EBF6F597]
2016-01-14 17:16:48,091 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: AC80E9F8F4F84A9E8F9E48C8EBF6F597 in vertex: CB83CB6CCE634AB68F06CC68C0E1D975
2016-01-14 17:16:48,091 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: CB83CB6CCE634AB68F06CC68C0E1D975 with 1 source tasks and 1 pending tasks
2016-01-14 17:16:48,091 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:48,091 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791807913_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791807913_0001_1_01, startRequestedTime=1452791808090, startedTime=1452791808090
2016-01-14 17:16:48,092 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791807913_0001_1_01 [CB83CB6CCE634AB68F06CC68C0E1D975] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:48,092 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [BF7A219A04154B7AB49FC3B3EC9ACD7A/AD016E9AFC194B9EB0836320381C40C0] every/(1/1) ...platform/ALL_UNKNOWN_SWAP, VertexName: AC80E9F8F4F84A9E8F9E48C8EBF6F597, VertexParallelism: 1, TaskAttemptID:attempt_1452791807913_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=757CC289630D4EE6A47A3645029D7DEA, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CB83CB6CCE634AB68F06CC68C0E1D975, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:48,092 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791807913_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:48,092 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:48,093 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791807913_0001, with dag id: dag_1452791807913_0001_1
2016-01-14 17:16:48,093 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:48,094 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:48,094 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:48,094 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791807913_0001_00_000001
2016-01-14 17:16:48,095 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791807913_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:48,096 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791807913_0001_00_000001
2016-01-14 17:16:48,096 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:48,097 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791807913_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791807913_0001_00_000001, launchTime=1452791808097
2016-01-14 17:16:48,097 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:48,097 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791807913_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:48,097 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791807913_0001_00_000001
2016-01-14 17:16:48,098 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791807913_0001_1_00_000000_0] to container: [container_1452791807913_0001_00_000001]
2016-01-14 17:16:48,098 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791807913_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:48,098 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791807913_0001_00_000001 given task: attempt_1452791807913_0001_1_00_000000_0
2016-01-14 17:16:48,098 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791807913_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791807913_0001_1_00_000000_0
2016-01-14 17:16:48 Starting to run new task attempt: attempt_1452791807913_0001_1_00_000000_0
2016-01-14 17:16:48,099 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791807913_0001_1_00_000000_0] started. Is using containerId: [container_1452791807913_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:48,099 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791807913_0001_1_00_000000_0
2016-01-14 17:16:48,099 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:48,099 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:48,099 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:48,099 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:48,099 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:48,099 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:48,100 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [BF7A219A04154B7AB49FC3B3EC9ACD7A/AD016E9AFC194B9EB0836320381C40C0] every/(1/1) ...platform/ALL_UNKNOWN_SWAP, VertexName: AC80E9F8F4F84A9E8F9E48C8EBF6F597, VertexParallelism: 1, TaskAttemptID:attempt_1452791807913_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=757CC289630D4EE6A47A3645029D7DEA, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CB83CB6CCE634AB68F06CC68C0E1D975, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:48,100 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=720371712
2016-01-14 17:16:48,100 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791807913_0001_1_00_000000_0
2016-01-14 17:16:48,100 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:48,100 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:48,099 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791807913_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=AC80E9F8F4F84A9E8F9E48C8EBF6F597, taskAttemptId=attempt_1452791807913_0001_1_00_000000_0, startTime=1452791808098, containerId=container_1452791807913_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791807913_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:48,099 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:48,101 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=757CC289630D4EE6A47A3645029D7DEA, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:48,101 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791807913_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:48,101 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791807913_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:48,101 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:48,102 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 757CC289630D4EE6A47A3645029D7DEA
2016-01-14 17:16:48,102 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:48,102 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 757CC289630D4EE6A47A3645029D7DEA
2016-01-14 17:16:48,102 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791807913_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:48,104 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:48,104 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 757CC289630D4EE6A47A3645029D7DEA
2016-01-14 17:16:48,105 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=CB83CB6CCE634AB68F06CC68C0E1D975, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:48,104 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:48,105 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:48,105 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: CB83CB6CCE634AB68F06CC68C0E1D975
2016-01-14 17:16:48,106 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:48,107 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: CB83CB6CCE634AB68F06CC68C0E1D975
2016-01-14 17:16:48,107 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: CB83CB6CCE634AB68F06CC68C0E1D975
2016-01-14 17:16:48,105 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 757CC289630D4EE6A47A3645029D7DEA
2016-01-14 17:16:48,109 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: AC80E9F8F4F84A9E8F9E48C8EBF6F597, ordinal: 0
2016-01-14 17:16:48,109 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AC80E9F8F4F84A9E8F9E48C8EBF6F597, mem on start (mb), free: 244, total: 687, max: 687
2016-01-14 17:16:48,109 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:48,109 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:48,109 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:48,109 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:48,109 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:48,109 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:48,109 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:48,110 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:48,110 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:48,110 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 720371712, TotalAvailable: 482649047, TotalRequested/TotalJVMHeap:0.15
2016-01-14 17:16:48,110 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 757CC289630D4EE6A47A3645029D7DEA, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:48,110 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, CB83CB6CCE634AB68F06CC68C0E1D975, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:48,110 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:48,111 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 757CC289630D4EE6A47A3645029D7DEA being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:48,111 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:48,111 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:48,111 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 757CC289630D4EE6A47A3645029D7DEA
2016-01-14 17:16:48,111 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:48,112 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 757CC289630D4EE6A47A3645029D7DEA
2016-01-14 17:16:48,112 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:48,112 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791807913_0001_1_00_000000_0
2016-01-14 17:16:48,112 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:48,113 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@5a84c54b
2016-01-14 17:16:48,114 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@5a84c54b
2016-01-14 17:16:48,115 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:48,116 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:48,115 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:48,116 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 757CC289630D4EE6A47A3645029D7DEA
2016-01-14 17:16:48,117 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 65CD87A5D22842F4B4AD750DD2D5B314
2016-01-14 17:16:48,117 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 65CD87A5D22842F4B4AD750DD2D5B314
2016-01-14 17:16:48,117 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@66359092; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@46a695aa; comparator=cascading.tuple.hadoop.util.TupleComparator@62abf022; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:48,118 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:48,118 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:48,118 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:48,132 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:48,132 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:48,132 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:48,132 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:48,133 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:48,133 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:48,133 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:48,134 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: AC80E9F8F4F84A9E8F9E48C8EBF6F597, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:48,136 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791807913_0001_1_00_000000_0
2016-01-14 17:16:48,136 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AC80E9F8F4F84A9E8F9E48C8EBF6F597, mem on close (mb), free: 142, total: 687, max: 687
2016-01-14 17:16:48,136 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:48,137 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:48,137 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:48,137 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:48,139 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:48 Completed running task attempt: attempt_1452791807913_0001_1_00_000000_0
2016-01-14 17:16:48,142 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791807913_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:48,143 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791807913_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791807913_0001_1_00, eventCount=1
2016-01-14 17:16:48,143 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=720371712, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791808134, Process_Duration=2, Process_End_Time=1452791808136, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:48,143 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791807913_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=AC80E9F8F4F84A9E8F9E48C8EBF6F597, taskAttemptId=attempt_1452791807913_0001_1_00_000000_0, startTime=1452791808098, finishTime=1452791808143, timeTaken=45, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=720371712, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791808134, Process_Duration=2, Process_End_Time=1452791808136, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:48,143 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791807913_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:48,143 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:48,144 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:48,143 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:48,144 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:48,144 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791807913_0001_1_00_000000_0
2016-01-14 17:16:48,144 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791807913_0001_1][Event:TASK_FINISHED]: vertexName=AC80E9F8F4F84A9E8F9E48C8EBF6F597, taskId=task_1452791807913_0001_1_00_000000, startTime=1452791808098, finishTime=1452791808144, timeTaken=46, status=SUCCEEDED, successfulAttemptID=attempt_1452791807913_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=720371712, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791808134, Process_Duration=2, Process_End_Time=1452791808136, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:48,144 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791807913_0001_00_000001
2016-01-14 17:16:48,144 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791807913_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:48,144 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:48,144 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791807913_0001_1_00 [AC80E9F8F4F84A9E8F9E48C8EBF6F597] : 1
2016-01-14 17:16:48,144 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791807913_0001_1_00 [AC80E9F8F4F84A9E8F9E48C8EBF6F597]
2016-01-14 17:16:48,145 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:48,145 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791807913_0001_1][Event:VERTEX_FINISHED]: vertexName=AC80E9F8F4F84A9E8F9E48C8EBF6F597, vertexId=vertex_1452791807913_0001_1_00, initRequestedTime=1452791808075, initedTime=1452791808087, startRequestedTime=1452791808088, startedTime=1452791808088, finishTime=1452791808144, timeTaken=56, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=720371712, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791808134, Process_Duration=2, Process_End_Time=1452791808136, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791808098, firstTasksToStart=[ task_1452791807913_0001_1_00_000000 ], lastTaskFinishTime=1452791808143, lastTasksToFinish=[ task_1452791807913_0001_1_00_000000 ], minTaskDuration=45, maxTaskDuration=45, avgTaskDuration=45.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791807913_0001_1_00_000000 ], longestDurationTasks=[ task_1452791807913_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:48,145 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791807913_0001_1_00 [AC80E9F8F4F84A9E8F9E48C8EBF6F597] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:48,145 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791807913_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:48,145 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791807913_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791807913_0001_00_000001, stoppedTime=1452791808145, exitStatus=0
2016-01-14 17:16:48,146 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:48,146 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791807913_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:48,146 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791807913_0001_1_01 [CB83CB6CCE634AB68F06CC68C0E1D975] attempt: attempt_1452791807913_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:48,146 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: CB83CB6CCE634AB68F06CC68C0E1D975
2016-01-14 17:16:48,146 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791807913_0001_1_01 [CB83CB6CCE634AB68F06CC68C0E1D975]
2016-01-14 17:16:48,146 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791807913_0001_1_00 [AC80E9F8F4F84A9E8F9E48C8EBF6F597] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:48,146 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791807913_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:48,146 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791807913_0001_1][Event:TASK_STARTED]: vertexName=CB83CB6CCE634AB68F06CC68C0E1D975, taskId=task_1452791807913_0001_1_01_000000, scheduledTime=1452791808146, launchTime=1452791808146
2016-01-14 17:16:48,146 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791807913_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:48,146 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:48,147 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791807913_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:48,147 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791807913_0001_1_01 [CB83CB6CCE634AB68F06CC68C0E1D975]
2016-01-14 17:16:48,147 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [BF7A219A04154B7AB49FC3B3EC9ACD7A/AD016E9AFC194B9EB0836320381C40C0] every/(1/1) ...platform/ALL_UNKNOWN_SWAP, VertexName: CB83CB6CCE634AB68F06CC68C0E1D975, VertexParallelism: 1, TaskAttemptID:attempt_1452791807913_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=AC80E9F8F4F84A9E8F9E48C8EBF6F597, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=D360597DC62F4DA887D53D81F871AF68, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:48,148 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791807913_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:48,148 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:48,149 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791807913_0001_00_000002
2016-01-14 17:16:48,149 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791807913_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:48,150 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791807913_0001_00_000002
2016-01-14 17:16:48,150 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:48,151 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791807913_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:48,151 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791807913_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791807913_0001_00_000002, launchTime=1452791808151
2016-01-14 17:16:48,151 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:48,152 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791807913_0001_00_000002
2016-01-14 17:16:48,152 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791807913_0001_1_01_000000_0] to container: [container_1452791807913_0001_00_000002]
2016-01-14 17:16:48,152 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791807913_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:48,152 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791807913_0001_00_000002 given task: attempt_1452791807913_0001_1_01_000000_0
2016-01-14 17:16:48,152 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791807913_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791807913_0001_1_01_000000_0
2016-01-14 17:16:48 Starting to run new task attempt: attempt_1452791807913_0001_1_01_000000_0
2016-01-14 17:16:48,152 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791807913_0001_1_01_000000_0] started. Is using containerId: [container_1452791807913_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:48,152 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791807913_0001_1_01_000000_0
2016-01-14 17:16:48,153 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:48,153 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791807913_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CB83CB6CCE634AB68F06CC68C0E1D975, taskAttemptId=attempt_1452791807913_0001_1_01_000000_0, startTime=1452791808152, containerId=container_1452791807913_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791807913_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:48,153 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791807913_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:48,153 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:48,153 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791807913_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:48,153 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:48,153 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:48,153 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:48,153 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:48,154 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:48,154 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [BF7A219A04154B7AB49FC3B3EC9ACD7A/AD016E9AFC194B9EB0836320381C40C0] every/(1/1) ...platform/ALL_UNKNOWN_SWAP, VertexName: CB83CB6CCE634AB68F06CC68C0E1D975, VertexParallelism: 1, TaskAttemptID:attempt_1452791807913_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=AC80E9F8F4F84A9E8F9E48C8EBF6F597, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=D360597DC62F4DA887D53D81F871AF68, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:48,154 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=720371712
2016-01-14 17:16:48,154 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791807913_0001_1_01_000000_0
2016-01-14 17:16:48,154 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:48,154 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:48,155 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791807913_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:48,155 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=AC80E9F8F4F84A9E8F9E48C8EBF6F597, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:48,155 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791807913_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:48,155 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:48,156 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: AC80E9F8F4F84A9E8F9E48C8EBF6F597
2016-01-14 17:16:48,155 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=D360597DC62F4DA887D53D81F871AF68, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:48,155 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:48,155 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791807913_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:48,156 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791807913_0001_00_000001, Exiting
2016-01-14 17:16:48,156 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791807913_0001_00_000001
2016-01-14 17:16:48,156 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791807913_0001_00_000001
2016-01-14 17:16:48,157 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791807913_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:48,157 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791807913_0001_00_000001 completed successfully
2016-01-14 17:16:48,158 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791807913_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:48,158 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:48,158 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: D360597DC62F4DA887D53D81F871AF68
2016-01-14 17:16:48,158 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:48,158 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: D360597DC62F4DA887D53D81F871AF68
2016-01-14 17:16:48,158 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:48,162 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CB83CB6CCE634AB68F06CC68C0E1D975, ordinal: 1
2016-01-14 17:16:48,162 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CB83CB6CCE634AB68F06CC68C0E1D975, mem on start (mb), free: 138, total: 687, max: 687
2016-01-14 17:16:48,162 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:48,163 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:48,164 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 648334528, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:48,164 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:48,164 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 648334528, type: INPUT, componentVertexName: AC80E9F8F4F84A9E8F9E48C8EBF6F597
2016-01-14 17:16:48,164 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: AC80E9F8F4F84A9E8F9E48C8EBF6F597
2016-01-14 17:16:48,165 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:48,169 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:48,169 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: D360597DC62F4DA887D53D81F871AF68
2016-01-14 17:16:48,169 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:48,169 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:48,169 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:48,169 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:48,169 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:48,169 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:48,169 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 648334528, TotalRequestedScaled: 5.984626412307693E8, TotalJVMHeap: 720371712, TotalAvailable: 482649047, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:48,170 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, D360597DC62F4DA887D53D81F871AF68, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:48,170 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, AC80E9F8F4F84A9E8F9E48C8EBF6F597, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=648334528, allocated=482649047
2016-01-14 17:16:48,170 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:48,170 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: AC80E9F8F4F84A9E8F9E48C8EBF6F597 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:48,170 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:48,170 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:48,170 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: AC80E9F8F4F84A9E8F9E48C8EBF6F597
2016-01-14 17:16:48,171 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:48,171 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:48,171 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: AC80E9F8F4F84A9E8F9E48C8EBF6F597 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:48,171 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=648334528, postMergeMem=0, RuntimeTotalAvailable=482649047. Updated to: ShuffleMem=482649047, postMergeMem: 0
2016-01-14 17:16:48,171 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=482649047, maxSingleShuffleLimit=120662264, mergeThreshold=434384128, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:48,172 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:48,172 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: AC80E9F8F4F84A9E8F9E48C8EBF6F597
2016-01-14 17:16:48,172 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:48,172 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791807913_0001_1_01_000000_0
2016-01-14 17:16:48,175 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:48,175 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 65CD87A5D22842F4B4AD750DD2D5B314
2016-01-14 17:16:48,175 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_SWAP"], id: D360597DC62F4DA887D53D81F871AF68
2016-01-14 17:16:48,175 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_SWAP"]
2016-01-14 17:16:48,176 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 65CD87A5D22842F4B4AD750DD2D5B314, for 1 inputs
2016-01-14 17:16:48,176 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791807913_0001_1_00_000000_0_10285, runDuration: 35587, ]
2016-01-14 17:16:48,177 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791807913_0001_1_00_000000_0_10285] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791808177, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 17:16:48,177 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : AC80E9F8F4F84A9E8F9E48C8EBF6F597
2016-01-14 17:16:48,177 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:48,177 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [AC80E9F8F4F84A9E8F9E48C8EBF6F597] #48 in 1ms
2016-01-14 17:16:48,178 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [AC80E9F8F4F84A9E8F9E48C8EBF6F597] #48, status:false, isInterrupted:false
2016-01-14 17:16:48,178 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:48,179 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:48,179 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:48,179 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:48,179 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:48,179 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : AC80E9F8F4F84A9E8F9E48C8EBF6F597
2016-01-14 17:16:48,180 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CB83CB6CCE634AB68F06CC68C0E1D975, all 1 inputs ready in: 00:00:00.004
2016-01-14 17:16:48,180 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:48,180 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@4f9adb93; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@3fc5b8b9
2016-01-14 17:16:48,182 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:48,182 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:48,183 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791807913_0001_1_01_000000_0
2016-01-14 17:16:48,183 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791807913_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:48,183 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791807913_0001_1_01_000000_0
2016-01-14 17:16:48,183 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CB83CB6CCE634AB68F06CC68C0E1D975, mem on close (mb), free: 135, total: 687, max: 687
2016-01-14 17:16:48,184 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: AC80E9F8F4F84A9E8F9E48C8EBF6F597
2016-01-14 17:16:48,184 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:48,184 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:48,184 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:48,184 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:48,184 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:48 Completed running task attempt: attempt_1452791807913_0001_1_01_000000_0
2016-01-14 17:16:48,185 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791807913_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:48,185 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=652, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=720371712, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791808180, Process_Duration=3, Process_End_Time=1452791808183, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 17:16:48,185 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:48,185 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791807913_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CB83CB6CCE634AB68F06CC68C0E1D975, taskAttemptId=attempt_1452791807913_0001_1_01_000000_0, startTime=1452791808152, finishTime=1452791808185, timeTaken=33, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=652, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=720371712, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791808180, Process_Duration=3, Process_End_Time=1452791808183, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:48,186 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791807913_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:48,186 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:48,186 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:48,186 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791807913_0001_1_01_000000_0
2016-01-14 17:16:48,187 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791807913_0001_00_000002
2016-01-14 17:16:48,187 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:48,187 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791807913_0001_1][Event:TASK_FINISHED]: vertexName=CB83CB6CCE634AB68F06CC68C0E1D975, taskId=task_1452791807913_0001_1_01_000000, startTime=1452791808152, finishTime=1452791808187, timeTaken=35, status=SUCCEEDED, successfulAttemptID=attempt_1452791807913_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=652, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=720371712, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791808180, Process_Duration=3, Process_End_Time=1452791808183, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 17:16:48,187 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:48,187 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791807913_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:48,188 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791807913_0001_1_01 [CB83CB6CCE634AB68F06CC68C0E1D975] : 1
2016-01-14 17:16:48,188 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791807913_0001_1_01 [CB83CB6CCE634AB68F06CC68C0E1D975]
2016-01-14 17:16:48,189 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791807913_0001_1][Event:VERTEX_FINISHED]: vertexName=CB83CB6CCE634AB68F06CC68C0E1D975, vertexId=vertex_1452791807913_0001_1_01, initRequestedTime=1452791808077, initedTime=1452791808084, startRequestedTime=1452791808090, startedTime=1452791808090, finishTime=1452791808188, timeTaken=98, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=652, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=720371712, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791808180, Process_Duration=3, Process_End_Time=1452791808183, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452791808152, firstTasksToStart=[ task_1452791807913_0001_1_01_000000 ], lastTaskFinishTime=1452791808185, lastTasksToFinish=[ task_1452791807913_0001_1_01_000000 ], minTaskDuration=33, maxTaskDuration=33, avgTaskDuration=33.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791807913_0001_1_01_000000 ], longestDurationTasks=[ task_1452791807913_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:48,189 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791807913_0001_1_01 [CB83CB6CCE634AB68F06CC68C0E1D975] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:48,189 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791807913_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:48,189 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791807913_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791807913_0001_00_000002, stoppedTime=1452791808189, exitStatus=0
2016-01-14 17:16:48,189 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791807913_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:48,189 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:48,189 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791807913_0001_1_01 [CB83CB6CCE634AB68F06CC68C0E1D975] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:48,189 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791807913_0001_1
2016-01-14 17:16:48,190 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791807913_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452791807913_0001_1
2016-01-14 17:16:48,190 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: D360597DC62F4DA887D53D81F871AF68 for vertex: vertex_1452791807913_0001_1_01 [CB83CB6CCE634AB68F06CC68C0E1D975]
2016-01-14 17:16:48,190 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:48,197 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452791807913_0001_1_00 [AC80E9F8F4F84A9E8F9E48C8EBF6F597]
2016-01-14 17:16:48,198 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791807913_0001_1, queueSize=0
2016-01-14 17:16:48,198 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791807913_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 17:16:48,198 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791807913_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:48,198 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791807913_0001_00_000002, Exiting
2016-01-14 17:16:48,198 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791807913_0001_00_000002
2016-01-14 17:16:48,198 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791807913_0001_00_000002
2016-01-14 17:16:48,198 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:48,199 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791807913_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:48,199 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791807913_0001_00_000002 completed successfully
2016-01-14 17:16:48,199 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791807913_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791807913_0001_1, startTime=1452791808075, finishTime=1452791808197, timeTaken=122, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=1505, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1440743424, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905583616314, Process_Duration=5, Process_End_Time=2905583616319, Read_Duration=1, Tuples_Read=40, Tuples_Written=40, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 17:16:48,199 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791807913_0001_1 finished with state: SUCCEEDED
2016-01-14 17:16:48,200 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791807913_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:48,200 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452791807913_0001_00_000002
2016-01-14 17:16:48,200 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791807913_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:16:48 Completed Dag: dag_1452791807913_0001_1
2016-01-14 17:16:48,200 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791807913_0001_1, dagState=SUCCEEDED
2016-01-14 17:16:48,200 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791807913_0001_1_post
2016-01-14 17:16:48,201 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:48,201 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:48,205 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...platform/ALL_UNKNOWN_SWAP, applicationId=application_1452791807913_0001
2016-01-14 17:16:48,207 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:48,207 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:48,207 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:48,207 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:48,207 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:48,207 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:48,207 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:48,208 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:48,208 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:48,208 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:48,209 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:48,209 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:48,210 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:48,213 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/BF7A219A04154B7AB49FC3B3EC9ACD7A/AD016E9AFC194B9EB0836320381C40C0/.tez/application_1452791807913_0001
2016-01-14 17:16:48,214 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.318
2016-01-14 17:16:48,463 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_UNKNOWN_SWAP
2016-01-14 17:16:48,482 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:48,483 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:48,512 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.005
2016-01-14 17:16:48,517 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-01-14 17:16:48,517 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:48,517 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:48,517 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:48,529 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:48,533 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_-1
2016-01-14 17:16:48,572 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 17:16:48,572 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 17:16:48,572 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:48,573 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:48,573 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_-1"]
2016-01-14 17:16:48,573 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:48,573 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:48,574 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:48,574 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...blyplatform/ALL_VALUES_-1
2016-01-14 17:16:48,574 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:48,585 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:48,590 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:48,591 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:48,591 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:48,600 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3C5D25B203454ED3B12C5FB2839E87D5/B5DF274B794E4AAF9DF8BD54A96957FB/.tez/application_1452791808590_0001 doesn't exist and is created
2016-01-14 17:16:48,615 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:48,615 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:48,616 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3C5D25B203454ED3B12C5FB2839E87D5/B5DF274B794E4AAF9DF8BD54A96957FB/.tez/application_1452791808590_0001
2016-01-14 17:16:48,619 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791808590_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:48,630 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:48,631 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:48,632 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:48,632 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:48,632 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3C5D25B203454ED3B12C5FB2839E87D5/B5DF274B794E4AAF9DF8BD54A96957FB/.tez/application_1452791808590_0001/localmode-log-dir
2016-01-14 17:16:48,632 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3C5D25B203454ED3B12C5FB2839E87D5/B5DF274B794E4AAF9DF8BD54A96957FB/.tez/application_1452791808590_0001/localmode-log-dir/history.txt.appattempt_1452791808590_0001_000000, maxErrors=10
2016-01-14 17:16:48,633 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:48,633 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791808590_0001_000000, appSubmitTime=1452791808618, launchTime=1452791808619
2016-01-14 17:16:48,636 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:48,636 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:48,638 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:48,639 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:48,640 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:50929
2016-01-14 17:16:48,641 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:48,643 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791808590_0001_000000, startTime=1452791808643
2016-01-14 17:16:48,643 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:48,643 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:48,715 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:48,716 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:48,716 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...blyplatform/ALL_VALUES_-1, applicationId=application_1452791808590_0001, dagName=[3C5D25B203454ED3B12C5FB2839E87D5/B5DF274B794E4AAF9DF8BD54A96957FB] every/(1/1) ...blyplatform/ALL_VALUES_-1
2016-01-14 17:16:48,730 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [3C5D25B203454ED3B12C5FB2839E87D5/B5DF274B794E4AAF9DF8BD54A96957FB] every/(1/1) ...blyplatform/ALL_VALUES_-1
2016-01-14 17:16:48,731 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791808590_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3C5D25B203454ED3B12C5FB2839E87D5/B5DF274B794E4AAF9DF8BD54A96957FB/.tez/application_1452791808590_0001/localmode-log-dir/dag_1452791808590_0001_1.dot
2016-01-14 17:16:48,731 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3C5D25B203454ED3B12C5FB2839E87D5/B5DF274B794E4AAF9DF8BD54A96957FB/.tez/application_1452791808590_0001/localmode-log-dir/dag_1452791808590_0001_1-tez-dag.pb.txt
2016-01-14 17:16:48,737 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791808590_0001_1
2016-01-14 17:16:48,737 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:48,737 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [3C5D25B203454ED3B12C5FB2839E87D5/B5DF274B794E4AAF9DF8BD54A96957FB] every/(1/1) ...blyplatform/ALL_VALUES_-1
2016-01-14 17:16:48 Running Dag: dag_1452791808590_0001_1
2016-01-14 17:16:48,747 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:48,748 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791808590_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791808590_0001_1, submitTime=1452791808730
2016-01-14 17:16:48,748 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex D3B35B437E304BE58A0687ACA6B803B8
2016-01-14 17:16:48,748 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:48,748 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:48,749 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791808590_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791808590_0001_1, initTime=1452791808748
2016-01-14 17:16:48,749 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791808590_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:48,749 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791808590_0001_1][Event:DAG_STARTED]: dagID=dag_1452791808590_0001_1, startTime=1452791808749
2016-01-14 17:16:48,749 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:48,749 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791808590_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:48,749 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: F605DDA5A74A41A8988F4065CA241C79 : {3850DE501A814D309CDCFA15DA2A4294={InputName=3850DE501A814D309CDCFA15DA2A4294}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:48,749 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 3850DE501A814D309CDCFA15DA2A4294, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:48,749 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791808590_0001_1_00 [F605DDA5A74A41A8988F4065CA241C79]
2016-01-14 17:16:48,749 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...blyplatform/ALL_VALUES_-1, applicationId=application_1452791808590_0001, dagName=[3C5D25B203454ED3B12C5FB2839E87D5/B5DF274B794E4AAF9DF8BD54A96957FB] every/(1/1) ...blyplatform/ALL_VALUES_-1
2016-01-14 17:16:48,749 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791808590_0001_1_00 [F605DDA5A74A41A8988F4065CA241C79]
2016-01-14 17:16:48,749 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791808590_0001_1_00 [F605DDA5A74A41A8988F4065CA241C79]
2016-01-14 17:16:48,750 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791808590_0001_1_00 [F605DDA5A74A41A8988F4065CA241C79]. Starting root input initializers: 1
2016-01-14 17:16:48,750 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:48,750 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791808590_0001_1_00 [F605DDA5A74A41A8988F4065CA241C79] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:48,750 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791808590_0001_1_01 [D3B35B437E304BE58A0687ACA6B803B8]
2016-01-14 17:16:48,750 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 3850DE501A814D309CDCFA15DA2A4294 on vertex vertex_1452791808590_0001_1_00 [F605DDA5A74A41A8988F4065CA241C79]
2016-01-14 17:16:48,753 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:48,753 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:48,753 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791808590_0001_1_01 [D3B35B437E304BE58A0687ACA6B803B8]
2016-01-14 17:16:48,754 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791808590_0001_1_01 [D3B35B437E304BE58A0687ACA6B803B8]
2016-01-14 17:16:48,754 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791808590_0001_1_01 [D3B35B437E304BE58A0687ACA6B803B8]
2016-01-14 17:16:48,754 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=6F306006C4FB426A8D6FC2059448034B, vertexId=vertex_1452791808590_0001_1_01 [D3B35B437E304BE58A0687ACA6B803B8], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:48,754 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=6F306006C4FB426A8D6FC2059448034B, vertexId=vertex_1452791808590_0001_1_01 [D3B35B437E304BE58A0687ACA6B803B8]
2016-01-14 17:16:48,753 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 3850DE501A814D309CDCFA15DA2A4294 asking for 1 tasks. Headroom: 732 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:48,755 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=6F306006C4FB426A8D6FC2059448034B, vertexName=D3B35B437E304BE58A0687ACA6B803B8, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:48,756 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=6F306006C4FB426A8D6FC2059448034B, vertexId=vertex_1452791808590_0001_1_01 [D3B35B437E304BE58A0687ACA6B803B8]
2016-01-14 17:16:48,756 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:48,756 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:48,757 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:48,757 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791808590_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D3B35B437E304BE58A0687ACA6B803B8, vertexId=vertex_1452791808590_0001_1_01, initRequestedTime=1452791808750, initedTime=1452791808757, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:48,757 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791808590_0001_1_01 [D3B35B437E304BE58A0687ACA6B803B8] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:48,758 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:48,758 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 3850DE501A814D309CDCFA15DA2A4294 on vertex vertex_1452791808590_0001_1_00 [F605DDA5A74A41A8988F4065CA241C79]
2016-01-14 17:16:48,759 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791808590_0001_1_00 [F605DDA5A74A41A8988F4065CA241C79] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:48,759 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {3850DE501A814D309CDCFA15DA2A4294=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:48,759 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791808590_0001_1_00 [F605DDA5A74A41A8988F4065CA241C79] parallelism set to 1
2016-01-14 17:16:48,759 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791808590_0001_1_00 [F605DDA5A74A41A8988F4065CA241C79] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:48,759 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791808590_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791808590_0001_1_00, eventCount=1
2016-01-14 17:16:48,759 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791808590_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F605DDA5A74A41A8988F4065CA241C79, vertexId=vertex_1452791808590_0001_1_00, initRequestedTime=1452791808749, initedTime=1452791808759, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:48,760 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791808590_0001_1_00 [F605DDA5A74A41A8988F4065CA241C79] with distanceFromRoot: 0
2016-01-14 17:16:48,760 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:48,760 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791808590_0001_1_00 [F605DDA5A74A41A8988F4065CA241C79] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:48,760 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in F605DDA5A74A41A8988F4065CA241C79
2016-01-14 17:16:48,760 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791808590_0001_1_00 [F605DDA5A74A41A8988F4065CA241C79]
2016-01-14 17:16:48,761 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:48,761 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791808590_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791808590_0001_1_00, startRequestedTime=1452791808760, startedTime=1452791808760
2016-01-14 17:16:48,761 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791808590_0001_1_00 [F605DDA5A74A41A8988F4065CA241C79] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:48,761 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:48,762 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791808590_0001_1][Event:TASK_STARTED]: vertexName=F605DDA5A74A41A8988F4065CA241C79, taskId=task_1452791808590_0001_1_00_000000, scheduledTime=1452791808761, launchTime=1452791808761
2016-01-14 17:16:48,762 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791808590_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:48,762 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791808590_0001_1_00 for vertex: vertex_1452791808590_0001_1_01 [D3B35B437E304BE58A0687ACA6B803B8] numStartedSources: 1 numSources: 1
2016-01-14 17:16:48,762 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791808590_0001_1_01 [D3B35B437E304BE58A0687ACA6B803B8] with distanceFromRoot: 1
2016-01-14 17:16:48,762 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791808590_0001_1_00 [F605DDA5A74A41A8988F4065CA241C79]
2016-01-14 17:16:48,763 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: F605DDA5A74A41A8988F4065CA241C79 in vertex: D3B35B437E304BE58A0687ACA6B803B8
2016-01-14 17:16:48,763 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: D3B35B437E304BE58A0687ACA6B803B8 with 1 source tasks and 1 pending tasks
2016-01-14 17:16:48,763 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:48,764 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791808590_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791808590_0001_1_01, startRequestedTime=1452791808763, startedTime=1452791808763
2016-01-14 17:16:48,764 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791808590_0001_1_01 [D3B35B437E304BE58A0687ACA6B803B8] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:48,764 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [3C5D25B203454ED3B12C5FB2839E87D5/B5DF274B794E4AAF9DF8BD54A96957FB] every/(1/1) ...blyplatform/ALL_VALUES_-1, VertexName: F605DDA5A74A41A8988F4065CA241C79, VertexParallelism: 1, TaskAttemptID:attempt_1452791808590_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=3850DE501A814D309CDCFA15DA2A4294, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D3B35B437E304BE58A0687ACA6B803B8, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:48,764 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791808590_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:48,764 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:48,765 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:48,765 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:48,765 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:48,766 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791808590_0001_00_000001
2016-01-14 17:16:48,766 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791808590_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:48,768 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791808590_0001_00_000001
2016-01-14 17:16:48,768 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:48,770 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791808590_0001, with dag id: dag_1452791808590_0001_1
2016-01-14 17:16:48,771 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:48,772 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791808590_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791808590_0001_00_000001, launchTime=1452791808771
2016-01-14 17:16:48,773 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791808590_0001_00_000001
2016-01-14 17:16:48,775 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791808590_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:48,784 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791808590_0001_1_00_000000_0] to container: [container_1452791808590_0001_00_000001]
2016-01-14 17:16:48,785 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791808590_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:48,785 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791808590_0001_00_000001 given task: attempt_1452791808590_0001_1_00_000000_0
2016-01-14 17:16:48,785 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791808590_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791808590_0001_1_00_000000_0
2016-01-14 17:16:48,785 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791808590_0001_1_00_000000_0] started. Is using containerId: [container_1452791808590_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:48 Starting to run new task attempt: attempt_1452791808590_0001_1_00_000000_0
2016-01-14 17:16:48,785 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791808590_0001_1_00_000000_0
2016-01-14 17:16:48,785 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:48,785 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:48,786 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:48,786 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:48,786 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:48,786 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:48,786 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:48,786 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [3C5D25B203454ED3B12C5FB2839E87D5/B5DF274B794E4AAF9DF8BD54A96957FB] every/(1/1) ...blyplatform/ALL_VALUES_-1, VertexName: F605DDA5A74A41A8988F4065CA241C79, VertexParallelism: 1, TaskAttemptID:attempt_1452791808590_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=3850DE501A814D309CDCFA15DA2A4294, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D3B35B437E304BE58A0687ACA6B803B8, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:48,786 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791808590_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F605DDA5A74A41A8988F4065CA241C79, taskAttemptId=attempt_1452791808590_0001_1_00_000000_0, startTime=1452791808785, containerId=container_1452791808590_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791808590_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:48,786 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=767557632
2016-01-14 17:16:48,786 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791808590_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:48,786 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791808590_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:48,787 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791808590_0001_1_00_000000_0
2016-01-14 17:16:48,787 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:48,787 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:48,789 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791808590_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:48,789 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:48,789 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=D3B35B437E304BE58A0687ACA6B803B8, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:48,790 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:48,790 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: D3B35B437E304BE58A0687ACA6B803B8
2016-01-14 17:16:48,791 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:48,791 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: D3B35B437E304BE58A0687ACA6B803B8
2016-01-14 17:16:48,791 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: D3B35B437E304BE58A0687ACA6B803B8
2016-01-14 17:16:48,792 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:48,792 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=3850DE501A814D309CDCFA15DA2A4294, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:48,793 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:48,793 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 3850DE501A814D309CDCFA15DA2A4294
2016-01-14 17:16:48,794 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 3850DE501A814D309CDCFA15DA2A4294
2016-01-14 17:16:48,795 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F605DDA5A74A41A8988F4065CA241C79, ordinal: 0
2016-01-14 17:16:48,796 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F605DDA5A74A41A8988F4065CA241C79, mem on start (mb), free: 607, total: 732, max: 732
2016-01-14 17:16:48,796 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:48,796 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:48,796 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:48,796 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:48,796 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 3850DE501A814D309CDCFA15DA2A4294
2016-01-14 17:16:48,797 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 3850DE501A814D309CDCFA15DA2A4294
2016-01-14 17:16:48,797 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:48,797 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:48,797 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:48,798 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:48,798 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:48,798 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:48,798 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 767557632, TotalAvailable: 514263613, TotalRequested/TotalJVMHeap:0.14
2016-01-14 17:16:48,799 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, D3B35B437E304BE58A0687ACA6B803B8, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:48,799 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 3850DE501A814D309CDCFA15DA2A4294, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:48,799 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:48,800 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 3850DE501A814D309CDCFA15DA2A4294 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:48,800 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 3850DE501A814D309CDCFA15DA2A4294
2016-01-14 17:16:48,801 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:48,801 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 3850DE501A814D309CDCFA15DA2A4294
2016-01-14 17:16:48,801 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:48,802 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:48,802 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:48,802 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791808590_0001_1_00_000000_0
2016-01-14 17:16:48,804 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:48,804 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@375b9e1c
2016-01-14 17:16:48,805 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:48,805 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@375b9e1c
2016-01-14 17:16:48,805 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:48,805 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:48,805 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 3850DE501A814D309CDCFA15DA2A4294
2016-01-14 17:16:48,807 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 2822815925B64ACC815AB8EFC385A6E1
2016-01-14 17:16:48,807 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 2822815925B64ACC815AB8EFC385A6E1
2016-01-14 17:16:48,808 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@14fe5f60; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@4827a264; comparator=cascading.tuple.hadoop.util.TupleComparator@2216783e; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:48,808 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:48,808 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:48,808 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:48,822 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:48,822 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:48,822 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:48,822 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:48,822 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:48,824 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:48,824 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:48,825 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F605DDA5A74A41A8988F4065CA241C79, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:48,828 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791808590_0001_1_00_000000_0
2016-01-14 17:16:48,828 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F605DDA5A74A41A8988F4065CA241C79, mem on close (mb), free: 505, total: 732, max: 732
2016-01-14 17:16:48,828 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:48,828 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:48,828 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:48,828 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:48,830 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:48 Completed running task attempt: attempt_1452791808590_0001_1_00_000000_0
2016-01-14 17:16:48,833 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791808590_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:48,833 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791808590_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791808590_0001_1_00, eventCount=1
2016-01-14 17:16:48,833 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:48,833 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791808590_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F605DDA5A74A41A8988F4065CA241C79, taskAttemptId=attempt_1452791808590_0001_1_00_000000_0, startTime=1452791808785, finishTime=1452791808833, timeTaken=48, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=767557632, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791808825, Process_Duration=3, Process_End_Time=1452791808828, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:48,833 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791808590_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:48,833 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:48,833 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791808590_0001_1_00_000000_0
2016-01-14 17:16:48,834 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:48,834 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791808590_0001_1][Event:TASK_FINISHED]: vertexName=F605DDA5A74A41A8988F4065CA241C79, taskId=task_1452791808590_0001_1_00_000000, startTime=1452791808785, finishTime=1452791808834, timeTaken=49, status=SUCCEEDED, successfulAttemptID=attempt_1452791808590_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=767557632, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791808825, Process_Duration=3, Process_End_Time=1452791808828, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:48,834 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791808590_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:48,834 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791808590_0001_1_00 [F605DDA5A74A41A8988F4065CA241C79] : 1
2016-01-14 17:16:48,834 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=767557632, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791808825, Process_Duration=3, Process_End_Time=1452791808828, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:48,834 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791808590_0001_1_00 [F605DDA5A74A41A8988F4065CA241C79]
2016-01-14 17:16:48,835 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:48,835 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:48,835 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791808590_0001_00_000001
2016-01-14 17:16:48,835 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:48,835 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791808590_0001_1][Event:VERTEX_FINISHED]: vertexName=F605DDA5A74A41A8988F4065CA241C79, vertexId=vertex_1452791808590_0001_1_00, initRequestedTime=1452791808749, initedTime=1452791808759, startRequestedTime=1452791808760, startedTime=1452791808760, finishTime=1452791808835, timeTaken=75, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=767557632, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791808825, Process_Duration=3, Process_End_Time=1452791808828, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791808785, firstTasksToStart=[ task_1452791808590_0001_1_00_000000 ], lastTaskFinishTime=1452791808833, lastTasksToFinish=[ task_1452791808590_0001_1_00_000000 ], minTaskDuration=48, maxTaskDuration=48, avgTaskDuration=48.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791808590_0001_1_00_000000 ], longestDurationTasks=[ task_1452791808590_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:48,835 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791808590_0001_1_00 [F605DDA5A74A41A8988F4065CA241C79] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:48,835 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791808590_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:48,836 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791808590_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791808590_0001_00_000001, stoppedTime=1452791808835, exitStatus=0
2016-01-14 17:16:48,836 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791808590_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:48,836 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791808590_0001_1_01 [D3B35B437E304BE58A0687ACA6B803B8] attempt: attempt_1452791808590_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:48,836 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:48,836 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: D3B35B437E304BE58A0687ACA6B803B8
2016-01-14 17:16:48,836 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791808590_0001_1_01 [D3B35B437E304BE58A0687ACA6B803B8]
2016-01-14 17:16:48,836 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791808590_0001_1_00 [F605DDA5A74A41A8988F4065CA241C79] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:48,836 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791808590_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:48,837 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791808590_0001_1][Event:TASK_STARTED]: vertexName=D3B35B437E304BE58A0687ACA6B803B8, taskId=task_1452791808590_0001_1_01_000000, scheduledTime=1452791808836, launchTime=1452791808836
2016-01-14 17:16:48,837 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:48,837 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791808590_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:48,837 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791808590_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:48,837 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791808590_0001_1_01 [D3B35B437E304BE58A0687ACA6B803B8]
2016-01-14 17:16:48,837 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [3C5D25B203454ED3B12C5FB2839E87D5/B5DF274B794E4AAF9DF8BD54A96957FB] every/(1/1) ...blyplatform/ALL_VALUES_-1, VertexName: D3B35B437E304BE58A0687ACA6B803B8, VertexParallelism: 1, TaskAttemptID:attempt_1452791808590_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F605DDA5A74A41A8988F4065CA241C79, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=6F306006C4FB426A8D6FC2059448034B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:48,838 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791808590_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:48,838 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:48,838 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791808590_0001_00_000002
2016-01-14 17:16:48,838 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791808590_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:48,840 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791808590_0001_00_000002
2016-01-14 17:16:48,840 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:48,840 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791808590_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:48,840 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:48,840 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791808590_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791808590_0001_00_000002, launchTime=1452791808840
2016-01-14 17:16:48,841 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791808590_0001_00_000002
2016-01-14 17:16:48,841 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791808590_0001_1_01_000000_0] to container: [container_1452791808590_0001_00_000002]
2016-01-14 17:16:48,842 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791808590_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:48,842 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791808590_0001_00_000002 given task: attempt_1452791808590_0001_1_01_000000_0
2016-01-14 17:16:48,842 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791808590_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791808590_0001_1_01_000000_0
2016-01-14 17:16:48,842 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791808590_0001_1_01_000000_0] started. Is using containerId: [container_1452791808590_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:48 Starting to run new task attempt: attempt_1452791808590_0001_1_01_000000_0
2016-01-14 17:16:48,842 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791808590_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D3B35B437E304BE58A0687ACA6B803B8, taskAttemptId=attempt_1452791808590_0001_1_01_000000_0, startTime=1452791808842, containerId=container_1452791808590_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791808590_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:48,842 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791808590_0001_1_01_000000_0
2016-01-14 17:16:48,843 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:48,842 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:48,842 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791808590_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:48,843 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791808590_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:48,843 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:48,843 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:48,843 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:48,843 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:48,843 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:48,843 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [3C5D25B203454ED3B12C5FB2839E87D5/B5DF274B794E4AAF9DF8BD54A96957FB] every/(1/1) ...blyplatform/ALL_VALUES_-1, VertexName: D3B35B437E304BE58A0687ACA6B803B8, VertexParallelism: 1, TaskAttemptID:attempt_1452791808590_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F605DDA5A74A41A8988F4065CA241C79, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=6F306006C4FB426A8D6FC2059448034B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:48,843 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=767557632
2016-01-14 17:16:48,844 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791808590_0001_1_01_000000_0
2016-01-14 17:16:48,844 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:48,844 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791808590_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:48,844 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:48,845 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:48,845 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791808590_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:48,845 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791808590_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:48,845 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F605DDA5A74A41A8988F4065CA241C79, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:48,845 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791808590_0001_00_000001, Exiting
2016-01-14 17:16:48,845 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791808590_0001_00_000001
2016-01-14 17:16:48,845 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:48,845 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791808590_0001_00_000001
2016-01-14 17:16:48,846 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F605DDA5A74A41A8988F4065CA241C79
2016-01-14 17:16:48,846 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791808590_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:48,847 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791808590_0001_00_000001 completed successfully
2016-01-14 17:16:48,847 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791808590_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:48,847 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 690801856, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:48,847 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:48,847 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 690801856, type: INPUT, componentVertexName: F605DDA5A74A41A8988F4065CA241C79
2016-01-14 17:16:48,847 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F605DDA5A74A41A8988F4065CA241C79
2016-01-14 17:16:48,847 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:48,849 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=6F306006C4FB426A8D6FC2059448034B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:48,849 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:48,850 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 6F306006C4FB426A8D6FC2059448034B
2016-01-14 17:16:48,850 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:48,850 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 6F306006C4FB426A8D6FC2059448034B
2016-01-14 17:16:48,853 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D3B35B437E304BE58A0687ACA6B803B8, ordinal: 1
2016-01-14 17:16:48,853 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D3B35B437E304BE58A0687ACA6B803B8, mem on start (mb), free: 500, total: 732, max: 732
2016-01-14 17:16:48,853 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:48,853 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:48,853 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:48,858 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:48,858 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 6F306006C4FB426A8D6FC2059448034B
2016-01-14 17:16:48,858 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:48,859 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:48,859 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:48,859 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:48,859 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:48,859 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:48,859 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 690801856, TotalRequestedScaled: 6.376632516923077E8, TotalJVMHeap: 767557632, TotalAvailable: 514263613, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:48,860 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F605DDA5A74A41A8988F4065CA241C79, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=690801856, allocated=514263613
2016-01-14 17:16:48,860 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 6F306006C4FB426A8D6FC2059448034B, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:48,860 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:48,860 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F605DDA5A74A41A8988F4065CA241C79 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:48,860 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:48,860 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:48,860 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F605DDA5A74A41A8988F4065CA241C79
2016-01-14 17:16:48,861 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:48,861 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:48,861 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: F605DDA5A74A41A8988F4065CA241C79 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:48,861 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=690801856, postMergeMem=0, RuntimeTotalAvailable=514263613. Updated to: ShuffleMem=514263613, postMergeMem: 0
2016-01-14 17:16:48,862 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=514263613, maxSingleShuffleLimit=128565904, mergeThreshold=462837248, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:48,862 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:48,862 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F605DDA5A74A41A8988F4065CA241C79
2016-01-14 17:16:48,862 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:48,863 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791808590_0001_1_01_000000_0
2016-01-14 17:16:48,864 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791808590_0001_1_00_000000_0_10290, runDuration: 41967, ]
2016-01-14 17:16:48,865 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791808590_0001_1_00_000000_0_10290] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791808865, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 17:16:48,865 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : F605DDA5A74A41A8988F4065CA241C79
2016-01-14 17:16:48,865 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:48,866 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:48,866 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [F605DDA5A74A41A8988F4065CA241C79] #49 in 2ms
2016-01-14 17:16:48,866 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 2822815925B64ACC815AB8EFC385A6E1
2016-01-14 17:16:48,866 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [F605DDA5A74A41A8988F4065CA241C79] #49, status:false, isInterrupted:false
2016-01-14 17:16:48,866 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:48,866 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:48,866 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:48,867 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:48,867 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:48,867 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : F605DDA5A74A41A8988F4065CA241C79
2016-01-14 17:16:48,867 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:48,867 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_-1"], id: 6F306006C4FB426A8D6FC2059448034B
2016-01-14 17:16:48,867 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_-1"]
2016-01-14 17:16:48,868 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 2822815925B64ACC815AB8EFC385A6E1, for 1 inputs
2016-01-14 17:16:48,868 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D3B35B437E304BE58A0687ACA6B803B8, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:48,868 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@6a2e4036; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@279cfb0
2016-01-14 17:16:48,870 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:48,870 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(92)) - caught throwable
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:48,871 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:48,871 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:48,871 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791808590_0001_1_01_000000_0
2016-01-14 17:16:48,871 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791808590_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:48,872 INFO  task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1452791808590_0001_1_01_000000_0
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:48,873 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=767557632, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791808868, Process_Duration=4, Process_End_Time=1452791808872, Read_Duration=2, Tuples_Read=1]]
2016-01-14 17:16:48,873 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791808590_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D3B35B437E304BE58A0687ACA6B803B8, taskAttemptId=attempt_1452791808590_0001_1_01_000000_0, startTime=1452791808842, finishTime=1452791808873, timeTaken=31, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=767557632, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791808868, Process_Duration=4, Process_End_Time=1452791808872, Read_Duration=2, Tuples_Read=1
2016-01-14 17:16:48,873 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:48,873 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791808590_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-01-14 17:16:48,873 INFO  impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1452791808590_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-01-14 17:16:48,874 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791808590_0001_1][Event:TASK_FINISHED]: vertexName=D3B35B437E304BE58A0687ACA6B803B8, taskId=task_1452791808590_0001_1_01_000000, startTime=1452791808842, finishTime=1452791808874, timeTaken=32, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
], counters=Counters: 0
2016-01-14 17:16:48,874 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:48,873 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D3B35B437E304BE58A0687ACA6B803B8, mem on close (mb), free: 497, total: 732, max: 732
2016-01-14 17:16:48,874 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=D3B35B437E304BE58A0687ACA6B803B8, index=1
2016-01-14 17:16:48,874 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791808590_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-01-14 17:16:48,874 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: F605DDA5A74A41A8988F4065CA241C79
2016-01-14 17:16:48,874 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791808590_0001_1_01 [D3B35B437E304BE58A0687ACA6B803B8] : 1
2016-01-14 17:16:48,874 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:48,874 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:48,874 INFO  impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1452791808590_0001_1_01 [D3B35B437E304BE58A0687ACA6B803B8] because task failed: task_1452791808590_0001_1_01_000000
2016-01-14 17:16:48,875 INFO  impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1452791808590_0001_1_01 [D3B35B437E304BE58A0687ACA6B803B8] due to trigger: OWN_TASK_FAILURE
2016-01-14 17:16:48,875 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:48,875 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:48,875 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:48,875 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:48,875 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=D3B35B437E304BE58A0687ACA6B803B8, sourceVertex=F605DDA5A74A41A8988F4065CA241C79
2016-01-14 17:16:48,875 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-01-14 17:16:48,876 INFO  impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1452791808590_0001_1_01 [D3B35B437E304BE58A0687ACA6B803B8]
2016-01-14 17:16:48,876 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=D3B35B437E304BE58A0687ACA6B803B8, sourceVertex=6F306006C4FB426A8D6FC2059448034B
2016-01-14 17:16:48,876 INFO  impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=6F306006C4FB426A8D6FC2059448034B, vertexId=vertex_1452791808590_0001_1_01 [D3B35B437E304BE58A0687ACA6B803B8]
2016-01-14 17:16:48,876 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791808590_0001_00_000002
2016-01-14 17:16:48,876 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791808590_0001_00_000002
2016-01-14 17:16:48,877 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791808590_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-01-14 17:16:48,877 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1452791808590_0001_00_000002 completed but with errors
2016-01-14 17:16:48,877 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:48,877 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791808590_0001_1][Event:VERTEX_FINISHED]: vertexName=D3B35B437E304BE58A0687ACA6B803B8, vertexId=vertex_1452791808590_0001_1_01, initRequestedTime=1452791808750, initedTime=1452791808757, startRequestedTime=1452791808763, startedTime=1452791808763, finishTime=1452791808877, timeTaken=114, status=FAILED, diagnostics=Task failed, taskId=task_1452791808590_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1452791808590_0001_1_01 [D3B35B437E304BE58A0687ACA6B803B8] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1452791808842, firstTasksToStart=[ task_1452791808590_0001_1_01_000000 ], lastTaskFinishTime=1452791808873, lastTasksToFinish=[ task_1452791808590_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[  ], longestDurationTasks=[  ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-01-14 17:16:48,878 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791808590_0001_1_01 [D3B35B437E304BE58A0687ACA6B803B8] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-01-14 17:16:48,878 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791808590_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791808590_0001_00_000002, stoppedTime=1452791808878, exitStatus=0
2016-01-14 17:16:48,878 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:48,878 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791808590_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:48,878 INFO  node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1452791808590_0001_1_01_000000_0 failed: true container: container_1452791808590_0001_00_000002 numFailedTAs: 0
2016-01-14 17:16:48,878 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791808590_0001_1_01 [D3B35B437E304BE58A0687ACA6B803B8] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-01-14 17:16:48,878 INFO  impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-01-14 17:16:48,878 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791808590_0001_1
2016-01-14 17:16:48,878 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 6F306006C4FB426A8D6FC2059448034B for vertex: vertex_1452791808590_0001_1_01 [D3B35B437E304BE58A0687ACA6B803B8]
2016-01-14 17:16:48,879 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1452791808590_0001_1_00 [F605DDA5A74A41A8988F4065CA241C79]
2016-01-14 17:16:48,880 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791808590_0001_1, queueSize=0
2016-01-14 17:16:48,880 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:48,881 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791808590_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791808590_0001_1, startTime=1452791808749, finishTime=1452791808879, timeTaken=130, status=FAILED, diagnostics=Vertex failed, vertexName=D3B35B437E304BE58A0687ACA6B803B8, vertexId=vertex_1452791808590_0001_1_01, diagnostics=[Task failed, taskId=task_1452791808590_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1452791808590_0001_1_01 [D3B35B437E304BE58A0687ACA6B803B8] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=767557632, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791808825, Process_Duration=3, Process_End_Time=1452791808828, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:48,881 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791808590_0001_1 finished with state: FAILED
2016-01-14 17:16:48,881 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791808590_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:48,881 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1452791808590_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-01-14 17:16:48,881 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791808590_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:16:48,881 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452791808590_0001_00_000002
2016-01-14 17:16:48 Completed Dag: dag_1452791808590_0001_1
2016-01-14 17:16:48,882 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791808590_0001_1, dagState=FAILED
2016-01-14 17:16:48,882 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791808590_0001_1_post
2016-01-14 17:16:48,882 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:48,882 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:48,882 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791808590_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-01-14 17:16:48,893 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...blyplatform/ALL_VALUES_-1, applicationId=application_1452791808590_0001
2016-01-14 17:16:48,894 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:48,894 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:48,894 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:48,895 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:48,895 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:48,895 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:48,895 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:48,895 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:48,895 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:48,895 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:48,896 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:48,896 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:48,896 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:48,900 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3C5D25B203454ED3B12C5FB2839E87D5/B5DF274B794E4AAF9DF8BD54A96957FB/.tez/application_1452791808590_0001
2016-01-14 17:16:48,901 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping all jobs
2016-01-14 17:16:48,901 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping: (1/1) ...blyplatform/ALL_VALUES_-1
2016-01-14 17:16:48,902 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopped all jobs
2016-01-14 17:16:48,902 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.330
2016-01-14 17:16:48,902 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_VALUES_-1
2016-01-14 17:16:48,927 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:48,927 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:48,952 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.002
2016-01-14 17:16:48,953 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.003
2016-01-14 17:16:48,953 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 17:16:48,954 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 17:16:48,954 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_VALUES_'field'
2016-01-14 17:16:48,976 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:48,977 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:49,004 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.005
2016-01-14 17:16:49,014 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-01-14 17:16:49,015 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:49,015 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:49,016 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:49,028 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:49,032 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_RESULTS
2016-01-14 17:16:49,071 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:49,071 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:49,072 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_RESULTS"]
2016-01-14 17:16:49,072 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:49,072 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:49,072 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:49,072 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...atform/ALL_VALUES_RESULTS
2016-01-14 17:16:49,072 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:49,080 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 17:16:49,080 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 17:16:49,083 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:49,088 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:49,088 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:49,088 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:49,097 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/0DECFFB599A7443D94CC065AD4D2D4D5/BAD1EE3803F04DD8AF07867A17C606D6/.tez/application_1452791809088_0001 doesn't exist and is created
2016-01-14 17:16:49,111 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:49,112 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:49,112 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/0DECFFB599A7443D94CC065AD4D2D4D5/BAD1EE3803F04DD8AF07867A17C606D6/.tez/application_1452791809088_0001
2016-01-14 17:16:49,114 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791809088_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:49,124 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:49,124 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:49,125 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:49,125 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:49,125 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/0DECFFB599A7443D94CC065AD4D2D4D5/BAD1EE3803F04DD8AF07867A17C606D6/.tez/application_1452791809088_0001/localmode-log-dir
2016-01-14 17:16:49,126 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/0DECFFB599A7443D94CC065AD4D2D4D5/BAD1EE3803F04DD8AF07867A17C606D6/.tez/application_1452791809088_0001/localmode-log-dir/history.txt.appattempt_1452791809088_0001_000000, maxErrors=10
2016-01-14 17:16:49,126 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:49,126 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791809088_0001_000000, appSubmitTime=1452791809114, launchTime=1452791809114
2016-01-14 17:16:49,130 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:39794
2016-01-14 17:16:49,131 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:49,131 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:49,131 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:49,132 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:49,133 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:49,134 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791809088_0001_000000, startTime=1452791809134
2016-01-14 17:16:49,134 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:49,134 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:49,212 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:49,212 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:49,212 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...atform/ALL_VALUES_RESULTS, applicationId=application_1452791809088_0001, dagName=[0DECFFB599A7443D94CC065AD4D2D4D5/BAD1EE3803F04DD8AF07867A17C606D6] every/(1/1) ...atform/ALL_VALUES_RESULTS
2016-01-14 17:16:49,226 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [0DECFFB599A7443D94CC065AD4D2D4D5/BAD1EE3803F04DD8AF07867A17C606D6] every/(1/1) ...atform/ALL_VALUES_RESULTS
2016-01-14 17:16:49,227 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791809088_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/0DECFFB599A7443D94CC065AD4D2D4D5/BAD1EE3803F04DD8AF07867A17C606D6/.tez/application_1452791809088_0001/localmode-log-dir/dag_1452791809088_0001_1.dot
2016-01-14 17:16:49,228 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/0DECFFB599A7443D94CC065AD4D2D4D5/BAD1EE3803F04DD8AF07867A17C606D6/.tez/application_1452791809088_0001/localmode-log-dir/dag_1452791809088_0001_1-tez-dag.pb.txt
2016-01-14 17:16:49,234 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791809088_0001_1
2016-01-14 17:16:49,234 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:49,235 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [0DECFFB599A7443D94CC065AD4D2D4D5/BAD1EE3803F04DD8AF07867A17C606D6] every/(1/1) ...atform/ALL_VALUES_RESULTS
2016-01-14 17:16:49 Running Dag: dag_1452791809088_0001_1
2016-01-14 17:16:49,246 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:49,247 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809088_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791809088_0001_1, submitTime=1452791809226
2016-01-14 17:16:49,248 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex D14AEC2BBE3741DF8EF08F1A159864E6
2016-01-14 17:16:49,248 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:49,248 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:49,249 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809088_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791809088_0001_1, initTime=1452791809247
2016-01-14 17:16:49,249 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791809088_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:49,249 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809088_0001_1][Event:DAG_STARTED]: dagID=dag_1452791809088_0001_1, startTime=1452791809249
2016-01-14 17:16:49,249 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:49,249 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:49,250 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791809088_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:49,250 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: B30B7966CCFA42E899F392DCDCB5ACEB : {0C52F10225204FDAA06DD7729197828C={InputName=0C52F10225204FDAA06DD7729197828C}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:49,250 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 0C52F10225204FDAA06DD7729197828C, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:49,250 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791809088_0001_1_00 [B30B7966CCFA42E899F392DCDCB5ACEB]
2016-01-14 17:16:49,250 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791809088_0001_1_00 [B30B7966CCFA42E899F392DCDCB5ACEB]
2016-01-14 17:16:49,250 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791809088_0001_1_00 [B30B7966CCFA42E899F392DCDCB5ACEB]
2016-01-14 17:16:49,250 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791809088_0001_1_00 [B30B7966CCFA42E899F392DCDCB5ACEB]. Starting root input initializers: 1
2016-01-14 17:16:49,251 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791809088_0001_1_00 [B30B7966CCFA42E899F392DCDCB5ACEB] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:49,251 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791809088_0001_1_01 [D14AEC2BBE3741DF8EF08F1A159864E6]
2016-01-14 17:16:49,252 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...atform/ALL_VALUES_RESULTS, applicationId=application_1452791809088_0001, dagName=[0DECFFB599A7443D94CC065AD4D2D4D5/BAD1EE3803F04DD8AF07867A17C606D6] every/(1/1) ...atform/ALL_VALUES_RESULTS
2016-01-14 17:16:49,254 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:49,254 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791809088_0001_1_01 [D14AEC2BBE3741DF8EF08F1A159864E6]
2016-01-14 17:16:49,254 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791809088_0001_1_01 [D14AEC2BBE3741DF8EF08F1A159864E6]
2016-01-14 17:16:49,254 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791809088_0001_1_01 [D14AEC2BBE3741DF8EF08F1A159864E6]
2016-01-14 17:16:49,254 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=364B4E2264BD4F82A6729BC52D906BA1, vertexId=vertex_1452791809088_0001_1_01 [D14AEC2BBE3741DF8EF08F1A159864E6], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:49,255 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=364B4E2264BD4F82A6729BC52D906BA1, vertexId=vertex_1452791809088_0001_1_01 [D14AEC2BBE3741DF8EF08F1A159864E6]
2016-01-14 17:16:49,256 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=364B4E2264BD4F82A6729BC52D906BA1, vertexName=D14AEC2BBE3741DF8EF08F1A159864E6, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:49,256 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=364B4E2264BD4F82A6729BC52D906BA1, vertexId=vertex_1452791809088_0001_1_01 [D14AEC2BBE3741DF8EF08F1A159864E6]
2016-01-14 17:16:49,258 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:49,258 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 0C52F10225204FDAA06DD7729197828C on vertex vertex_1452791809088_0001_1_00 [B30B7966CCFA42E899F392DCDCB5ACEB]
2016-01-14 17:16:49,259 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:49,259 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 0C52F10225204FDAA06DD7729197828C asking for 1 tasks. Headroom: 732 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:49,260 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:49,260 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:49,261 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:49,261 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 0C52F10225204FDAA06DD7729197828C on vertex vertex_1452791809088_0001_1_00 [B30B7966CCFA42E899F392DCDCB5ACEB]
2016-01-14 17:16:49,258 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809088_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D14AEC2BBE3741DF8EF08F1A159864E6, vertexId=vertex_1452791809088_0001_1_01, initRequestedTime=1452791809251, initedTime=1452791809257, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:49,261 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791809088_0001_1_01 [D14AEC2BBE3741DF8EF08F1A159864E6] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:49,261 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791809088_0001_1_00 [B30B7966CCFA42E899F392DCDCB5ACEB] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:49,262 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {0C52F10225204FDAA06DD7729197828C=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:49,262 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791809088_0001_1_00 [B30B7966CCFA42E899F392DCDCB5ACEB] parallelism set to 1
2016-01-14 17:16:49,262 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791809088_0001_1_00 [B30B7966CCFA42E899F392DCDCB5ACEB] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:49,262 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809088_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791809088_0001_1_00, eventCount=1
2016-01-14 17:16:49,262 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809088_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B30B7966CCFA42E899F392DCDCB5ACEB, vertexId=vertex_1452791809088_0001_1_00, initRequestedTime=1452791809250, initedTime=1452791809262, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:49,262 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791809088_0001_1_00 [B30B7966CCFA42E899F392DCDCB5ACEB] with distanceFromRoot: 0
2016-01-14 17:16:49,262 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:49,262 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791809088_0001_1_00 [B30B7966CCFA42E899F392DCDCB5ACEB] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:49,263 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in B30B7966CCFA42E899F392DCDCB5ACEB
2016-01-14 17:16:49,263 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791809088_0001_1_00 [B30B7966CCFA42E899F392DCDCB5ACEB]
2016-01-14 17:16:49,263 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809088_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791809088_0001_1_00, startRequestedTime=1452791809263, startedTime=1452791809263
2016-01-14 17:16:49,263 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:49,263 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791809088_0001_1_00 [B30B7966CCFA42E899F392DCDCB5ACEB] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:49,263 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:49,263 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809088_0001_1][Event:TASK_STARTED]: vertexName=B30B7966CCFA42E899F392DCDCB5ACEB, taskId=task_1452791809088_0001_1_00_000000, scheduledTime=1452791809263, launchTime=1452791809263
2016-01-14 17:16:49,264 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791809088_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:49,264 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791809088_0001_1_00 for vertex: vertex_1452791809088_0001_1_01 [D14AEC2BBE3741DF8EF08F1A159864E6] numStartedSources: 1 numSources: 1
2016-01-14 17:16:49,264 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791809088_0001_1_01 [D14AEC2BBE3741DF8EF08F1A159864E6] with distanceFromRoot: 1
2016-01-14 17:16:49,264 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791809088_0001_1_00 [B30B7966CCFA42E899F392DCDCB5ACEB]
2016-01-14 17:16:49,264 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: B30B7966CCFA42E899F392DCDCB5ACEB in vertex: D14AEC2BBE3741DF8EF08F1A159864E6
2016-01-14 17:16:49,264 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: D14AEC2BBE3741DF8EF08F1A159864E6 with 1 source tasks and 1 pending tasks
2016-01-14 17:16:49,265 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:49,265 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809088_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791809088_0001_1_01, startRequestedTime=1452791809264, startedTime=1452791809264
2016-01-14 17:16:49,265 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791809088_0001_1_01 [D14AEC2BBE3741DF8EF08F1A159864E6] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:49,265 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [0DECFFB599A7443D94CC065AD4D2D4D5/BAD1EE3803F04DD8AF07867A17C606D6] every/(1/1) ...atform/ALL_VALUES_RESULTS, VertexName: B30B7966CCFA42E899F392DCDCB5ACEB, VertexParallelism: 1, TaskAttemptID:attempt_1452791809088_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=0C52F10225204FDAA06DD7729197828C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D14AEC2BBE3741DF8EF08F1A159864E6, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:49,265 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791809088_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:49,266 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:49,266 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:49,266 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:49,267 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:49,267 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791809088_0001_00_000001
2016-01-14 17:16:49,267 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791809088_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:49,268 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791809088_0001_00_000001
2016-01-14 17:16:49,268 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:49,269 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791809088_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:49,269 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:49,270 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809088_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791809088_0001_00_000001, launchTime=1452791809269
2016-01-14 17:16:49,270 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791809088_0001_00_000001
2016-01-14 17:16:49,271 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791809088_0001_1_00_000000_0] to container: [container_1452791809088_0001_00_000001]
2016-01-14 17:16:49,271 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791809088_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:49,272 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791809088_0001_00_000001 given task: attempt_1452791809088_0001_1_00_000000_0
2016-01-14 17:16:49,272 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791809088_0001_1_00_000000_0] started. Is using containerId: [container_1452791809088_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:49,272 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809088_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B30B7966CCFA42E899F392DCDCB5ACEB, taskAttemptId=attempt_1452791809088_0001_1_00_000000_0, startTime=1452791809272, containerId=container_1452791809088_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791809088_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:49,272 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791809088_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:49,272 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791809088_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:49,272 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791809088_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791809088_0001_1_00_000000_0
2016-01-14 17:16:49,272 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:49 Starting to run new task attempt: attempt_1452791809088_0001_1_00_000000_0
2016-01-14 17:16:49,274 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791809088_0001_1_00_000000_0
2016-01-14 17:16:49,275 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:49,275 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791809088_0001, with dag id: dag_1452791809088_0001_1
2016-01-14 17:16:49,275 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:49,276 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:49,276 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:49,276 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:49,277 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:49,277 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [0DECFFB599A7443D94CC065AD4D2D4D5/BAD1EE3803F04DD8AF07867A17C606D6] every/(1/1) ...atform/ALL_VALUES_RESULTS, VertexName: B30B7966CCFA42E899F392DCDCB5ACEB, VertexParallelism: 1, TaskAttemptID:attempt_1452791809088_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=0C52F10225204FDAA06DD7729197828C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D14AEC2BBE3741DF8EF08F1A159864E6, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:49,277 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=767557632
2016-01-14 17:16:49,278 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791809088_0001_1_00_000000_0
2016-01-14 17:16:49,278 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791809088_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:49,278 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:49,279 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:49,279 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:49,281 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=0C52F10225204FDAA06DD7729197828C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:49,281 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:49,282 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 0C52F10225204FDAA06DD7729197828C
2016-01-14 17:16:49,282 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 0C52F10225204FDAA06DD7729197828C
2016-01-14 17:16:49,282 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=D14AEC2BBE3741DF8EF08F1A159864E6, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:49,282 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:49,283 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:49,285 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: D14AEC2BBE3741DF8EF08F1A159864E6
2016-01-14 17:16:49,284 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:49,285 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 0C52F10225204FDAA06DD7729197828C
2016-01-14 17:16:49,285 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 0C52F10225204FDAA06DD7729197828C
2016-01-14 17:16:49,286 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B30B7966CCFA42E899F392DCDCB5ACEB, ordinal: 0
2016-01-14 17:16:49,286 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B30B7966CCFA42E899F392DCDCB5ACEB, mem on start (mb), free: 605, total: 732, max: 732
2016-01-14 17:16:49,286 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:49,286 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:49,286 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:49,287 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:49,287 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: D14AEC2BBE3741DF8EF08F1A159864E6
2016-01-14 17:16:49,288 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: D14AEC2BBE3741DF8EF08F1A159864E6
2016-01-14 17:16:49,288 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:49,288 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:49,288 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:49,288 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:49,288 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:49,288 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:49,288 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 767557632, TotalAvailable: 514263613, TotalRequested/TotalJVMHeap:0.14
2016-01-14 17:16:49,288 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 0C52F10225204FDAA06DD7729197828C, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:49,288 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, D14AEC2BBE3741DF8EF08F1A159864E6, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:49,289 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:49,289 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 0C52F10225204FDAA06DD7729197828C being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:49,289 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:49,289 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:49,289 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 0C52F10225204FDAA06DD7729197828C
2016-01-14 17:16:49,289 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:49,289 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 0C52F10225204FDAA06DD7729197828C
2016-01-14 17:16:49,290 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:49,290 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791809088_0001_1_00_000000_0
2016-01-14 17:16:49,291 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:49,292 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@17d397d7
2016-01-14 17:16:49,292 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:49,293 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 0C52F10225204FDAA06DD7729197828C
2016-01-14 17:16:49,293 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: B7DF97F0314341B7AE889C38D6079140
2016-01-14 17:16:49,293 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] B7DF97F0314341B7AE889C38D6079140
2016-01-14 17:16:49,293 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@17d397d7
2016-01-14 17:16:49,293 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:49,294 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:49,293 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@64dd45d3; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@7b135360; comparator=cascading.tuple.hadoop.util.TupleComparator@31c702c0; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:49,294 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:49,295 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:49,295 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:49,309 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:49,309 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:49,309 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:49,310 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:49,310 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:49,310 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:49,311 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:49,311 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B30B7966CCFA42E899F392DCDCB5ACEB, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:49,313 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791809088_0001_1_00_000000_0
2016-01-14 17:16:49,313 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B30B7966CCFA42E899F392DCDCB5ACEB, mem on close (mb), free: 502, total: 732, max: 732
2016-01-14 17:16:49,314 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:49,314 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:49,314 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:49,314 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:49,316 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:49 Completed running task attempt: attempt_1452791809088_0001_1_00_000000_0
2016-01-14 17:16:49,319 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791809088_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:49,320 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809088_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791809088_0001_1_00, eventCount=1
2016-01-14 17:16:49,320 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=767557632, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791809311, Process_Duration=2, Process_End_Time=1452791809313, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:49,320 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:49,320 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809088_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B30B7966CCFA42E899F392DCDCB5ACEB, taskAttemptId=attempt_1452791809088_0001_1_00_000000_0, startTime=1452791809272, finishTime=1452791809320, timeTaken=48, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=767557632, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791809311, Process_Duration=2, Process_End_Time=1452791809313, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:49,320 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:49,321 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:49,321 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791809088_0001_00_000001
2016-01-14 17:16:49,321 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791809088_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:49,321 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791809088_0001_1_00_000000_0
2016-01-14 17:16:49,321 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:49,322 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809088_0001_1][Event:TASK_FINISHED]: vertexName=B30B7966CCFA42E899F392DCDCB5ACEB, taskId=task_1452791809088_0001_1_00_000000, startTime=1452791809272, finishTime=1452791809321, timeTaken=49, status=SUCCEEDED, successfulAttemptID=attempt_1452791809088_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=767557632, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791809311, Process_Duration=2, Process_End_Time=1452791809313, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:49,322 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791809088_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:49,322 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791809088_0001_1_00 [B30B7966CCFA42E899F392DCDCB5ACEB] : 1
2016-01-14 17:16:49,322 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791809088_0001_1_00 [B30B7966CCFA42E899F392DCDCB5ACEB]
2016-01-14 17:16:49,322 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:49,323 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809088_0001_1][Event:VERTEX_FINISHED]: vertexName=B30B7966CCFA42E899F392DCDCB5ACEB, vertexId=vertex_1452791809088_0001_1_00, initRequestedTime=1452791809250, initedTime=1452791809262, startRequestedTime=1452791809263, startedTime=1452791809263, finishTime=1452791809322, timeTaken=59, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=767557632, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791809311, Process_Duration=2, Process_End_Time=1452791809313, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791809272, firstTasksToStart=[ task_1452791809088_0001_1_00_000000 ], lastTaskFinishTime=1452791809320, lastTasksToFinish=[ task_1452791809088_0001_1_00_000000 ], minTaskDuration=48, maxTaskDuration=48, avgTaskDuration=48.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791809088_0001_1_00_000000 ], longestDurationTasks=[ task_1452791809088_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:49,323 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791809088_0001_1_00 [B30B7966CCFA42E899F392DCDCB5ACEB] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:49,323 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791809088_0001_1_01 [D14AEC2BBE3741DF8EF08F1A159864E6] attempt: attempt_1452791809088_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:49,323 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: D14AEC2BBE3741DF8EF08F1A159864E6
2016-01-14 17:16:49,323 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791809088_0001_1_01 [D14AEC2BBE3741DF8EF08F1A159864E6]
2016-01-14 17:16:49,323 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791809088_0001_1_00 [B30B7966CCFA42E899F392DCDCB5ACEB] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:49,324 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809088_0001_1][Event:TASK_STARTED]: vertexName=D14AEC2BBE3741DF8EF08F1A159864E6, taskId=task_1452791809088_0001_1_01_000000, scheduledTime=1452791809323, launchTime=1452791809323
2016-01-14 17:16:49,324 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:49,324 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:49,324 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791809088_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:49,325 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791809088_0001_1_01 [D14AEC2BBE3741DF8EF08F1A159864E6]
2016-01-14 17:16:49,325 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791809088_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:49,325 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809088_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791809088_0001_00_000001, stoppedTime=1452791809325, exitStatus=0
2016-01-14 17:16:49,325 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791809088_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:49,325 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [0DECFFB599A7443D94CC065AD4D2D4D5/BAD1EE3803F04DD8AF07867A17C606D6] every/(1/1) ...atform/ALL_VALUES_RESULTS, VertexName: D14AEC2BBE3741DF8EF08F1A159864E6, VertexParallelism: 1, TaskAttemptID:attempt_1452791809088_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B30B7966CCFA42E899F392DCDCB5ACEB, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=364B4E2264BD4F82A6729BC52D906BA1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:49,326 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791809088_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:49,326 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:49,326 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791809088_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:49,326 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:49,326 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791809088_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:49,327 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791809088_0001_00_000002
2016-01-14 17:16:49,327 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791809088_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:49,328 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791809088_0001_00_000002
2016-01-14 17:16:49,328 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:49,328 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791809088_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:49,329 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:49,329 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809088_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791809088_0001_00_000002, launchTime=1452791809328
2016-01-14 17:16:49,329 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791809088_0001_00_000002
2016-01-14 17:16:49,329 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791809088_0001_1_01_000000_0] to container: [container_1452791809088_0001_00_000002]
2016-01-14 17:16:49,329 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791809088_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:49,330 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791809088_0001_1_01_000000_0] started. Is using containerId: [container_1452791809088_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:49,330 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809088_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D14AEC2BBE3741DF8EF08F1A159864E6, taskAttemptId=attempt_1452791809088_0001_1_01_000000_0, startTime=1452791809330, containerId=container_1452791809088_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791809088_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:49,330 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:49,330 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791809088_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:49,330 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791809088_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:49,330 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791809088_0001_00_000002 given task: attempt_1452791809088_0001_1_01_000000_0
2016-01-14 17:16:49,330 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791809088_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791809088_0001_1_01_000000_0
2016-01-14 17:16:49 Starting to run new task attempt: attempt_1452791809088_0001_1_01_000000_0
2016-01-14 17:16:49,331 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791809088_0001_1_01_000000_0
2016-01-14 17:16:49,331 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:49,331 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:49,331 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:49,331 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:49,331 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:49,331 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:49,331 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [0DECFFB599A7443D94CC065AD4D2D4D5/BAD1EE3803F04DD8AF07867A17C606D6] every/(1/1) ...atform/ALL_VALUES_RESULTS, VertexName: D14AEC2BBE3741DF8EF08F1A159864E6, VertexParallelism: 1, TaskAttemptID:attempt_1452791809088_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B30B7966CCFA42E899F392DCDCB5ACEB, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=364B4E2264BD4F82A6729BC52D906BA1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:49,332 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=767557632
2016-01-14 17:16:49,331 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791809088_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:49,332 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791809088_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:49,332 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791809088_0001_1_01_000000_0
2016-01-14 17:16:49,332 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791809088_0001_00_000001, Exiting
2016-01-14 17:16:49,332 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791809088_0001_00_000001
2016-01-14 17:16:49,332 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791809088_0001_00_000001
2016-01-14 17:16:49,332 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:49,333 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:49,334 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791809088_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:49,335 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791809088_0001_00_000001 completed successfully
2016-01-14 17:16:49,335 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:49,335 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791809088_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:49,335 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B30B7966CCFA42E899F392DCDCB5ACEB, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:49,334 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791809088_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:49,336 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:49,336 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B30B7966CCFA42E899F392DCDCB5ACEB
2016-01-14 17:16:49,336 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=364B4E2264BD4F82A6729BC52D906BA1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:49,337 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:49,337 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 690801856, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:49,337 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:49,337 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 364B4E2264BD4F82A6729BC52D906BA1
2016-01-14 17:16:49,339 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:49,338 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:49,340 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 690801856, type: INPUT, componentVertexName: B30B7966CCFA42E899F392DCDCB5ACEB
2016-01-14 17:16:49,340 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B30B7966CCFA42E899F392DCDCB5ACEB
2016-01-14 17:16:49,340 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 364B4E2264BD4F82A6729BC52D906BA1
2016-01-14 17:16:49,342 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D14AEC2BBE3741DF8EF08F1A159864E6, ordinal: 1
2016-01-14 17:16:49,342 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D14AEC2BBE3741DF8EF08F1A159864E6, mem on start (mb), free: 498, total: 732, max: 732
2016-01-14 17:16:49,342 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:49,342 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:49,342 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:49,350 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:49,350 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 364B4E2264BD4F82A6729BC52D906BA1
2016-01-14 17:16:49,351 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:49,351 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:49,351 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:49,351 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:49,351 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:49,351 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:49,352 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 690801856, TotalRequestedScaled: 6.376632516923077E8, TotalJVMHeap: 767557632, TotalAvailable: 514263613, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:49,352 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B30B7966CCFA42E899F392DCDCB5ACEB, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=690801856, allocated=514263613
2016-01-14 17:16:49,352 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 364B4E2264BD4F82A6729BC52D906BA1, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:49,352 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:49,352 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B30B7966CCFA42E899F392DCDCB5ACEB being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:49,353 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B30B7966CCFA42E899F392DCDCB5ACEB
2016-01-14 17:16:49,353 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:49,353 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:49,353 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:49,353 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:49,353 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: B30B7966CCFA42E899F392DCDCB5ACEB with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:49,353 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=690801856, postMergeMem=0, RuntimeTotalAvailable=514263613. Updated to: ShuffleMem=514263613, postMergeMem: 0
2016-01-14 17:16:49,353 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=514263613, maxSingleShuffleLimit=128565904, mergeThreshold=462837248, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:49,354 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:49,354 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B30B7966CCFA42E899F392DCDCB5ACEB
2016-01-14 17:16:49,354 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:49,354 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791809088_0001_1_01_000000_0
2016-01-14 17:16:49,355 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791809088_0001_1_00_000000_0_10297, runDuration: 33244, ]
2016-01-14 17:16:49,357 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:49,357 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: B7DF97F0314341B7AE889C38D6079140
2016-01-14 17:16:49,358 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_RESULTS"], id: 364B4E2264BD4F82A6729BC52D906BA1
2016-01-14 17:16:49,358 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_RESULTS"]
2016-01-14 17:16:49,358 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] B7DF97F0314341B7AE889C38D6079140, for 1 inputs
2016-01-14 17:16:49,358 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791809088_0001_1_00_000000_0_10297] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791809358, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 17:16:49,359 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : B30B7966CCFA42E899F392DCDCB5ACEB
2016-01-14 17:16:49,359 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:49,359 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [B30B7966CCFA42E899F392DCDCB5ACEB] #50 in 2ms
2016-01-14 17:16:49,360 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [B30B7966CCFA42E899F392DCDCB5ACEB] #50, status:false, isInterrupted:false
2016-01-14 17:16:49,360 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:49,361 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:49,361 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:49,361 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:49,361 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:49,361 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : B30B7966CCFA42E899F392DCDCB5ACEB
2016-01-14 17:16:49,362 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:49,362 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D14AEC2BBE3741DF8EF08F1A159864E6, all 1 inputs ready in: 00:00:00.003
2016-01-14 17:16:49,362 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@5a8bb6b0; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@a870b41
2016-01-14 17:16:49,363 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:49,363 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(92)) - caught throwable
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:49,363 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:49,365 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:49,365 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791809088_0001_1_01_000000_0
2016-01-14 17:16:49,365 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791809088_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:49,366 INFO  task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1452791809088_0001_1_01_000000_0
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:49,367 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=767557632, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791809362, Process_Duration=4, Process_End_Time=1452791809366, Read_Duration=0, Tuples_Read=1]]
2016-01-14 17:16:49,367 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D14AEC2BBE3741DF8EF08F1A159864E6, mem on close (mb), free: 494, total: 732, max: 732
2016-01-14 17:16:49,367 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=D14AEC2BBE3741DF8EF08F1A159864E6, index=1
2016-01-14 17:16:49,367 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809088_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D14AEC2BBE3741DF8EF08F1A159864E6, taskAttemptId=attempt_1452791809088_0001_1_01_000000_0, startTime=1452791809330, finishTime=1452791809367, timeTaken=37, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=767557632, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791809362, Process_Duration=4, Process_End_Time=1452791809366, Read_Duration=0, Tuples_Read=1
2016-01-14 17:16:49,367 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:49,367 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791809088_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-01-14 17:16:49,367 INFO  impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1452791809088_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-01-14 17:16:49,368 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809088_0001_1][Event:TASK_FINISHED]: vertexName=D14AEC2BBE3741DF8EF08F1A159864E6, taskId=task_1452791809088_0001_1_01_000000, startTime=1452791809330, finishTime=1452791809367, timeTaken=37, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
], counters=Counters: 0
2016-01-14 17:16:49,367 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: B30B7966CCFA42E899F392DCDCB5ACEB
2016-01-14 17:16:49,368 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:49,368 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:49,368 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:49,368 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:49,368 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:49,368 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=D14AEC2BBE3741DF8EF08F1A159864E6, sourceVertex=B30B7966CCFA42E899F392DCDCB5ACEB
2016-01-14 17:16:49,368 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791809088_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-01-14 17:16:49,368 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:49,369 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791809088_0001_1_01 [D14AEC2BBE3741DF8EF08F1A159864E6] : 1
2016-01-14 17:16:49,369 INFO  impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1452791809088_0001_1_01 [D14AEC2BBE3741DF8EF08F1A159864E6] because task failed: task_1452791809088_0001_1_01_000000
2016-01-14 17:16:49,369 INFO  impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1452791809088_0001_1_01 [D14AEC2BBE3741DF8EF08F1A159864E6] due to trigger: OWN_TASK_FAILURE
2016-01-14 17:16:49,369 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:49,369 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=D14AEC2BBE3741DF8EF08F1A159864E6, sourceVertex=364B4E2264BD4F82A6729BC52D906BA1
2016-01-14 17:16:49,369 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-01-14 17:16:49,370 INFO  impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1452791809088_0001_1_01 [D14AEC2BBE3741DF8EF08F1A159864E6]
2016-01-14 17:16:49,370 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791809088_0001_00_000002
2016-01-14 17:16:49,370 INFO  impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=364B4E2264BD4F82A6729BC52D906BA1, vertexId=vertex_1452791809088_0001_1_01 [D14AEC2BBE3741DF8EF08F1A159864E6]
2016-01-14 17:16:49,370 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791809088_0001_00_000002
2016-01-14 17:16:49,370 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791809088_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-01-14 17:16:49,370 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1452791809088_0001_00_000002 completed but with errors
2016-01-14 17:16:49,371 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809088_0001_1][Event:VERTEX_FINISHED]: vertexName=D14AEC2BBE3741DF8EF08F1A159864E6, vertexId=vertex_1452791809088_0001_1_01, initRequestedTime=1452791809251, initedTime=1452791809257, startRequestedTime=1452791809264, startedTime=1452791809264, finishTime=1452791809371, timeTaken=107, status=FAILED, diagnostics=Task failed, taskId=task_1452791809088_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1452791809088_0001_1_01 [D14AEC2BBE3741DF8EF08F1A159864E6] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1452791809330, firstTasksToStart=[ task_1452791809088_0001_1_01_000000 ], lastTaskFinishTime=1452791809367, lastTasksToFinish=[ task_1452791809088_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[  ], longestDurationTasks=[  ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-01-14 17:16:49,371 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:49,371 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791809088_0001_1_01 [D14AEC2BBE3741DF8EF08F1A159864E6] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-01-14 17:16:49,372 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809088_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791809088_0001_00_000002, stoppedTime=1452791809372, exitStatus=0
2016-01-14 17:16:49,372 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791809088_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:49,372 INFO  node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1452791809088_0001_1_01_000000_0 failed: true container: container_1452791809088_0001_00_000002 numFailedTAs: 0
2016-01-14 17:16:49,372 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1452791809088_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-01-14 17:16:49,373 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791809088_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:16:49,373 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791809088_0001_1_01 [D14AEC2BBE3741DF8EF08F1A159864E6] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-01-14 17:16:49,373 INFO  impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-01-14 17:16:49,373 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791809088_0001_1
2016-01-14 17:16:49,373 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1452791809088_0001_1_00 [B30B7966CCFA42E899F392DCDCB5ACEB]
2016-01-14 17:16:49,373 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 364B4E2264BD4F82A6729BC52D906BA1 for vertex: vertex_1452791809088_0001_1_01 [D14AEC2BBE3741DF8EF08F1A159864E6]
2016-01-14 17:16:49,373 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:49,375 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791809088_0001_1, queueSize=0
2016-01-14 17:16:49,375 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:49,376 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809088_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791809088_0001_1, startTime=1452791809249, finishTime=1452791809374, timeTaken=125, status=FAILED, diagnostics=Vertex failed, vertexName=D14AEC2BBE3741DF8EF08F1A159864E6, vertexId=vertex_1452791809088_0001_1_01, diagnostics=[Task failed, taskId=task_1452791809088_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1452791809088_0001_1_01 [D14AEC2BBE3741DF8EF08F1A159864E6] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=767557632, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791809311, Process_Duration=2, Process_End_Time=1452791809313, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:49,377 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791809088_0001_1 finished with state: FAILED
2016-01-14 17:16:49,377 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791809088_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:49,378 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452791809088_0001_00_000002
2016-01-14 17:16:49,378 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791809088_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-01-14 17:16:49 Completed Dag: dag_1452791809088_0001_1
2016-01-14 17:16:49,379 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791809088_0001_1, dagState=FAILED
2016-01-14 17:16:49,380 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791809088_0001_1_post
2016-01-14 17:16:49,380 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:49,380 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:49,381 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...atform/ALL_VALUES_RESULTS, applicationId=application_1452791809088_0001
2016-01-14 17:16:49,383 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:49,383 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:49,383 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:49,383 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:49,383 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:49,383 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:49,384 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:49,384 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:49,384 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:49,384 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:49,384 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:49,385 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:49,385 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:49,389 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/0DECFFB599A7443D94CC065AD4D2D4D5/BAD1EE3803F04DD8AF07867A17C606D6/.tez/application_1452791809088_0001
2016-01-14 17:16:49,390 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping all jobs
2016-01-14 17:16:49,390 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping: (1/1) ...atform/ALL_VALUES_RESULTS
2016-01-14 17:16:49,390 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopped all jobs
2016-01-14 17:16:49,391 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.320
2016-01-14 17:16:49,391 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_VALUES_RESULTS
2016-01-14 17:16:49,419 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:49,419 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:49,447 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.005
2016-01-14 17:16:49,454 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-01-14 17:16:49,455 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:49,455 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:49,455 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:49,471 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:49,476 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_ALL
2016-01-14 17:16:49,517 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:49,517 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:49,517 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_ALL"]
2016-01-14 17:16:49,518 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:49,518 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:49,518 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:49,518 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...lyplatform/ALL_VALUES_ALL
2016-01-14 17:16:49,518 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:49,529 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:49,533 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:49,534 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:49,534 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:49,542 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/917E8AAA53F548CC897F96481F4AA76A/F3092A1E5B4F4130BB3E2185F1003973/.tez/application_1452791809534_0001 doesn't exist and is created
2016-01-14 17:16:49,557 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:49,558 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:49,558 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/917E8AAA53F548CC897F96481F4AA76A/F3092A1E5B4F4130BB3E2185F1003973/.tez/application_1452791809534_0001
2016-01-14 17:16:49,561 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791809534_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:49,571 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:49,572 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:49,573 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:49,573 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:49,573 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/917E8AAA53F548CC897F96481F4AA76A/F3092A1E5B4F4130BB3E2185F1003973/.tez/application_1452791809534_0001/localmode-log-dir
2016-01-14 17:16:49,574 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/917E8AAA53F548CC897F96481F4AA76A/F3092A1E5B4F4130BB3E2185F1003973/.tez/application_1452791809534_0001/localmode-log-dir/history.txt.appattempt_1452791809534_0001_000000, maxErrors=10
2016-01-14 17:16:49,574 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:49,574 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791809534_0001_000000, appSubmitTime=1452791809560, launchTime=1452791809560
2016-01-14 17:16:49,574 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 17:16:49,575 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 17:16:49,577 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:49,577 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:49,578 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:49,578 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:49,580 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:58170
2016-01-14 17:16:49,582 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:49,584 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791809534_0001_000000, startTime=1452791809584
2016-01-14 17:16:49,584 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:49,584 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:49,658 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:49,658 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:49,658 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...lyplatform/ALL_VALUES_ALL, applicationId=application_1452791809534_0001, dagName=[917E8AAA53F548CC897F96481F4AA76A/F3092A1E5B4F4130BB3E2185F1003973] every/(1/1) ...lyplatform/ALL_VALUES_ALL
2016-01-14 17:16:49,672 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [917E8AAA53F548CC897F96481F4AA76A/F3092A1E5B4F4130BB3E2185F1003973] every/(1/1) ...lyplatform/ALL_VALUES_ALL
2016-01-14 17:16:49,673 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791809534_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/917E8AAA53F548CC897F96481F4AA76A/F3092A1E5B4F4130BB3E2185F1003973/.tez/application_1452791809534_0001/localmode-log-dir/dag_1452791809534_0001_1.dot
2016-01-14 17:16:49,674 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/917E8AAA53F548CC897F96481F4AA76A/F3092A1E5B4F4130BB3E2185F1003973/.tez/application_1452791809534_0001/localmode-log-dir/dag_1452791809534_0001_1-tez-dag.pb.txt
2016-01-14 17:16:49,679 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791809534_0001_1
2016-01-14 17:16:49,679 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:49,679 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [917E8AAA53F548CC897F96481F4AA76A/F3092A1E5B4F4130BB3E2185F1003973] every/(1/1) ...lyplatform/ALL_VALUES_ALL
2016-01-14 17:16:49 Running Dag: dag_1452791809534_0001_1
2016-01-14 17:16:49,690 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809534_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791809534_0001_1, submitTime=1452791809672
2016-01-14 17:16:49,690 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:49,690 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex E25B12CAB49E47219630C22B77CE8A45
2016-01-14 17:16:49,690 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:49,691 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809534_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791809534_0001_1, initTime=1452791809690
2016-01-14 17:16:49,691 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791809534_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:49,691 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:49,691 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809534_0001_1][Event:DAG_STARTED]: dagID=dag_1452791809534_0001_1, startTime=1452791809691
2016-01-14 17:16:49,691 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:49,691 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791809534_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:49,691 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...lyplatform/ALL_VALUES_ALL, applicationId=application_1452791809534_0001, dagName=[917E8AAA53F548CC897F96481F4AA76A/F3092A1E5B4F4130BB3E2185F1003973] every/(1/1) ...lyplatform/ALL_VALUES_ALL
2016-01-14 17:16:49,691 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:49,691 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: BEE8090BA19646B99F5BFEAC84D7DFB7 : {27EF8314C9B047CF89B68870D1F5D1BE={InputName=27EF8314C9B047CF89B68870D1F5D1BE}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:49,692 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 27EF8314C9B047CF89B68870D1F5D1BE, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:49,692 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791809534_0001_1_00 [BEE8090BA19646B99F5BFEAC84D7DFB7]
2016-01-14 17:16:49,693 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791809534_0001_1_00 [BEE8090BA19646B99F5BFEAC84D7DFB7]
2016-01-14 17:16:49,693 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791809534_0001_1_00 [BEE8090BA19646B99F5BFEAC84D7DFB7]
2016-01-14 17:16:49,693 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791809534_0001_1_00 [BEE8090BA19646B99F5BFEAC84D7DFB7]. Starting root input initializers: 1
2016-01-14 17:16:49,693 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791809534_0001_1_00 [BEE8090BA19646B99F5BFEAC84D7DFB7] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:49,693 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791809534_0001_1_01 [E25B12CAB49E47219630C22B77CE8A45]
2016-01-14 17:16:49,695 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 27EF8314C9B047CF89B68870D1F5D1BE on vertex vertex_1452791809534_0001_1_00 [BEE8090BA19646B99F5BFEAC84D7DFB7]
2016-01-14 17:16:49,696 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:49,696 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791809534_0001_1_01 [E25B12CAB49E47219630C22B77CE8A45]
2016-01-14 17:16:49,696 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791809534_0001_1_01 [E25B12CAB49E47219630C22B77CE8A45]
2016-01-14 17:16:49,696 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791809534_0001_1_01 [E25B12CAB49E47219630C22B77CE8A45]
2016-01-14 17:16:49,696 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=57E9F985B3AD48C3947C2EAC78F5DB71, vertexId=vertex_1452791809534_0001_1_01 [E25B12CAB49E47219630C22B77CE8A45], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:49,696 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=57E9F985B3AD48C3947C2EAC78F5DB71, vertexId=vertex_1452791809534_0001_1_01 [E25B12CAB49E47219630C22B77CE8A45]
2016-01-14 17:16:49,698 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:49,698 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=57E9F985B3AD48C3947C2EAC78F5DB71, vertexName=E25B12CAB49E47219630C22B77CE8A45, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:49,698 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=57E9F985B3AD48C3947C2EAC78F5DB71, vertexId=vertex_1452791809534_0001_1_01 [E25B12CAB49E47219630C22B77CE8A45]
2016-01-14 17:16:49,698 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 27EF8314C9B047CF89B68870D1F5D1BE asking for 1 tasks. Headroom: 732 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:49,699 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:49,700 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:49,700 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:49,700 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809534_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E25B12CAB49E47219630C22B77CE8A45, vertexId=vertex_1452791809534_0001_1_01, initRequestedTime=1452791809693, initedTime=1452791809700, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:49,700 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791809534_0001_1_01 [E25B12CAB49E47219630C22B77CE8A45] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:49,701 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:49,701 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 27EF8314C9B047CF89B68870D1F5D1BE on vertex vertex_1452791809534_0001_1_00 [BEE8090BA19646B99F5BFEAC84D7DFB7]
2016-01-14 17:16:49,702 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791809534_0001_1_00 [BEE8090BA19646B99F5BFEAC84D7DFB7] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:49,702 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {27EF8314C9B047CF89B68870D1F5D1BE=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:49,702 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791809534_0001_1_00 [BEE8090BA19646B99F5BFEAC84D7DFB7] parallelism set to 1
2016-01-14 17:16:49,702 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791809534_0001_1_00 [BEE8090BA19646B99F5BFEAC84D7DFB7] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:49,702 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809534_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791809534_0001_1_00, eventCount=1
2016-01-14 17:16:49,703 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:49,703 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809534_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BEE8090BA19646B99F5BFEAC84D7DFB7, vertexId=vertex_1452791809534_0001_1_00, initRequestedTime=1452791809691, initedTime=1452791809703, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:49,703 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791809534_0001_1_00 [BEE8090BA19646B99F5BFEAC84D7DFB7] with distanceFromRoot: 0
2016-01-14 17:16:49,703 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791809534_0001_1_00 [BEE8090BA19646B99F5BFEAC84D7DFB7] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:49,703 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in BEE8090BA19646B99F5BFEAC84D7DFB7
2016-01-14 17:16:49,703 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791809534_0001_1_00 [BEE8090BA19646B99F5BFEAC84D7DFB7]
2016-01-14 17:16:49,704 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:49,704 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809534_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791809534_0001_1_00, startRequestedTime=1452791809703, startedTime=1452791809703
2016-01-14 17:16:49,704 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791809534_0001_1_00 [BEE8090BA19646B99F5BFEAC84D7DFB7] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:49,704 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:49,705 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809534_0001_1][Event:TASK_STARTED]: vertexName=BEE8090BA19646B99F5BFEAC84D7DFB7, taskId=task_1452791809534_0001_1_00_000000, scheduledTime=1452791809704, launchTime=1452791809704
2016-01-14 17:16:49,705 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791809534_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:49,705 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791809534_0001_1_00 for vertex: vertex_1452791809534_0001_1_01 [E25B12CAB49E47219630C22B77CE8A45] numStartedSources: 1 numSources: 1
2016-01-14 17:16:49,705 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791809534_0001_1_01 [E25B12CAB49E47219630C22B77CE8A45] with distanceFromRoot: 1
2016-01-14 17:16:49,705 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791809534_0001_1_00 [BEE8090BA19646B99F5BFEAC84D7DFB7]
2016-01-14 17:16:49,706 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: BEE8090BA19646B99F5BFEAC84D7DFB7 in vertex: E25B12CAB49E47219630C22B77CE8A45
2016-01-14 17:16:49,706 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: E25B12CAB49E47219630C22B77CE8A45 with 1 source tasks and 1 pending tasks
2016-01-14 17:16:49,706 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:49,706 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809534_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791809534_0001_1_01, startRequestedTime=1452791809705, startedTime=1452791809705
2016-01-14 17:16:49,706 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791809534_0001_1_01 [E25B12CAB49E47219630C22B77CE8A45] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:49,707 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [917E8AAA53F548CC897F96481F4AA76A/F3092A1E5B4F4130BB3E2185F1003973] every/(1/1) ...lyplatform/ALL_VALUES_ALL, VertexName: BEE8090BA19646B99F5BFEAC84D7DFB7, VertexParallelism: 1, TaskAttemptID:attempt_1452791809534_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=27EF8314C9B047CF89B68870D1F5D1BE, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E25B12CAB49E47219630C22B77CE8A45, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:49,707 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791809534_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:49,707 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:49,708 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:49,708 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:49,708 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:49,708 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791809534_0001, with dag id: dag_1452791809534_0001_1
2016-01-14 17:16:49,709 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791809534_0001_00_000001
2016-01-14 17:16:49,709 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791809534_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:49,710 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791809534_0001_00_000001
2016-01-14 17:16:49,710 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:49,711 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791809534_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:49,711 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:49,711 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809534_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791809534_0001_00_000001, launchTime=1452791809711
2016-01-14 17:16:49,712 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791809534_0001_00_000001
2016-01-14 17:16:49,712 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791809534_0001_1_00_000000_0] to container: [container_1452791809534_0001_00_000001]
2016-01-14 17:16:49,712 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791809534_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:49,713 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791809534_0001_1_00_000000_0] started. Is using containerId: [container_1452791809534_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:49,713 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809534_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BEE8090BA19646B99F5BFEAC84D7DFB7, taskAttemptId=attempt_1452791809534_0001_1_00_000000_0, startTime=1452791809713, containerId=container_1452791809534_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791809534_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:49,713 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:49,713 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791809534_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:49,713 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791809534_0001_00_000001 given task: attempt_1452791809534_0001_1_00_000000_0
2016-01-14 17:16:49,713 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791809534_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:49,713 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791809534_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791809534_0001_1_00_000000_0
2016-01-14 17:16:49 Starting to run new task attempt: attempt_1452791809534_0001_1_00_000000_0
2016-01-14 17:16:49,714 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791809534_0001_1_00_000000_0
2016-01-14 17:16:49,714 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:49,714 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:49,714 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:49,714 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:49,714 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:49,714 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:49,714 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [917E8AAA53F548CC897F96481F4AA76A/F3092A1E5B4F4130BB3E2185F1003973] every/(1/1) ...lyplatform/ALL_VALUES_ALL, VertexName: BEE8090BA19646B99F5BFEAC84D7DFB7, VertexParallelism: 1, TaskAttemptID:attempt_1452791809534_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=27EF8314C9B047CF89B68870D1F5D1BE, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E25B12CAB49E47219630C22B77CE8A45, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:49,715 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=768081920
2016-01-14 17:16:49,715 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791809534_0001_1_00_000000_0
2016-01-14 17:16:49,715 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:49,715 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:49,715 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791809534_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:49,716 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=27EF8314C9B047CF89B68870D1F5D1BE, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:49,716 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:49,716 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=E25B12CAB49E47219630C22B77CE8A45, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:49,717 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:49,717 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: E25B12CAB49E47219630C22B77CE8A45
2016-01-14 17:16:49,717 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:49,717 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 27EF8314C9B047CF89B68870D1F5D1BE
2016-01-14 17:16:49,718 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 27EF8314C9B047CF89B68870D1F5D1BE
2016-01-14 17:16:49,718 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:49,719 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:49,719 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: E25B12CAB49E47219630C22B77CE8A45
2016-01-14 17:16:49,719 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: E25B12CAB49E47219630C22B77CE8A45
2016-01-14 17:16:49,720 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:49,720 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 27EF8314C9B047CF89B68870D1F5D1BE
2016-01-14 17:16:49,720 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 27EF8314C9B047CF89B68870D1F5D1BE
2016-01-14 17:16:49,722 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BEE8090BA19646B99F5BFEAC84D7DFB7, ordinal: 0
2016-01-14 17:16:49,722 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BEE8090BA19646B99F5BFEAC84D7DFB7, mem on start (mb), free: 617, total: 732, max: 732
2016-01-14 17:16:49,722 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:49,722 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:49,722 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:49,722 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:49,722 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:49,723 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:49,723 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:49,723 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:49,723 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:49,723 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 768081920, TotalAvailable: 514614886, TotalRequested/TotalJVMHeap:0.14
2016-01-14 17:16:49,723 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 27EF8314C9B047CF89B68870D1F5D1BE, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:49,723 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, E25B12CAB49E47219630C22B77CE8A45, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:49,723 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:49,723 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 27EF8314C9B047CF89B68870D1F5D1BE being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:49,723 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:49,724 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:49,724 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 27EF8314C9B047CF89B68870D1F5D1BE
2016-01-14 17:16:49,724 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:49,724 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 27EF8314C9B047CF89B68870D1F5D1BE
2016-01-14 17:16:49,724 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:49,724 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791809534_0001_1_00_000000_0
2016-01-14 17:16:49,727 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:49,727 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 27EF8314C9B047CF89B68870D1F5D1BE
2016-01-14 17:16:49,727 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: C8CC15511B5948CA8C9E395855796812
2016-01-14 17:16:49,728 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] C8CC15511B5948CA8C9E395855796812
2016-01-14 17:16:49,728 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@4dfad807; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@f7cdb90; comparator=cascading.tuple.hadoop.util.TupleComparator@4e32d2e1; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:49,729 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:49,729 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:49,729 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:49,729 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:49,730 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@6ab2d9e2
2016-01-14 17:16:49,731 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@6ab2d9e2
2016-01-14 17:16:49,731 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:49,731 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:49,742 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:49,742 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:49,743 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:49,743 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:49,743 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:49,744 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:49,744 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:49,744 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BEE8090BA19646B99F5BFEAC84D7DFB7, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:49,747 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791809534_0001_1_00_000000_0
2016-01-14 17:16:49,747 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BEE8090BA19646B99F5BFEAC84D7DFB7, mem on close (mb), free: 515, total: 732, max: 732
2016-01-14 17:16:49,747 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:49,747 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:49,748 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:49,748 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:49,751 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:49 Completed running task attempt: attempt_1452791809534_0001_1_00_000000_0
2016-01-14 17:16:49,754 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791809534_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:49,755 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809534_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791809534_0001_1_00, eventCount=1
2016-01-14 17:16:49,755 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809534_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BEE8090BA19646B99F5BFEAC84D7DFB7, taskAttemptId=attempt_1452791809534_0001_1_00_000000_0, startTime=1452791809713, finishTime=1452791809755, timeTaken=42, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=768081920, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791809744, Process_Duration=3, Process_End_Time=1452791809747, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:49,755 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791809534_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:49,755 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:49,756 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:49,756 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791809534_0001_1_00_000000_0
2016-01-14 17:16:49,756 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:49,756 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809534_0001_1][Event:TASK_FINISHED]: vertexName=BEE8090BA19646B99F5BFEAC84D7DFB7, taskId=task_1452791809534_0001_1_00_000000, startTime=1452791809713, finishTime=1452791809756, timeTaken=43, status=SUCCEEDED, successfulAttemptID=attempt_1452791809534_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=768081920, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791809744, Process_Duration=3, Process_End_Time=1452791809747, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:49,756 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791809534_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:49,757 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791809534_0001_1_00 [BEE8090BA19646B99F5BFEAC84D7DFB7] : 1
2016-01-14 17:16:49,757 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791809534_0001_1_00 [BEE8090BA19646B99F5BFEAC84D7DFB7]
2016-01-14 17:16:49,757 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:49,757 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809534_0001_1][Event:VERTEX_FINISHED]: vertexName=BEE8090BA19646B99F5BFEAC84D7DFB7, vertexId=vertex_1452791809534_0001_1_00, initRequestedTime=1452791809691, initedTime=1452791809703, startRequestedTime=1452791809703, startedTime=1452791809703, finishTime=1452791809757, timeTaken=54, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=768081920, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791809744, Process_Duration=3, Process_End_Time=1452791809747, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791809713, firstTasksToStart=[ task_1452791809534_0001_1_00_000000 ], lastTaskFinishTime=1452791809755, lastTasksToFinish=[ task_1452791809534_0001_1_00_000000 ], minTaskDuration=42, maxTaskDuration=42, avgTaskDuration=42.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791809534_0001_1_00_000000 ], longestDurationTasks=[ task_1452791809534_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:49,757 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791809534_0001_1_00 [BEE8090BA19646B99F5BFEAC84D7DFB7] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:49,758 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791809534_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:49,758 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=768081920, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791809744, Process_Duration=3, Process_End_Time=1452791809747, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:49,758 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:49,758 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809534_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791809534_0001_00_000001, stoppedTime=1452791809758, exitStatus=0
2016-01-14 17:16:49,759 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791809534_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:49,759 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:49,759 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:49,759 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791809534_0001_1_01 [E25B12CAB49E47219630C22B77CE8A45] attempt: attempt_1452791809534_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:49,759 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: E25B12CAB49E47219630C22B77CE8A45
2016-01-14 17:16:49,759 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791809534_0001_1_01 [E25B12CAB49E47219630C22B77CE8A45]
2016-01-14 17:16:49,759 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791809534_0001_1_00 [BEE8090BA19646B99F5BFEAC84D7DFB7] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:49,759 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791809534_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:49,759 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791809534_0001_00_000001
2016-01-14 17:16:49,760 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791809534_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:49,760 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791809534_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:49,760 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791809534_0001_00_000001, Exiting
2016-01-14 17:16:49,760 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:49,760 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791809534_0001_00_000001
2016-01-14 17:16:49,760 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791809534_0001_00_000001
2016-01-14 17:16:49,760 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809534_0001_1][Event:TASK_STARTED]: vertexName=E25B12CAB49E47219630C22B77CE8A45, taskId=task_1452791809534_0001_1_01_000000, scheduledTime=1452791809760, launchTime=1452791809760
2016-01-14 17:16:49,761 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791809534_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:49,761 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791809534_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:49,761 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791809534_0001_1_01 [E25B12CAB49E47219630C22B77CE8A45]
2016-01-14 17:16:49,761 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [917E8AAA53F548CC897F96481F4AA76A/F3092A1E5B4F4130BB3E2185F1003973] every/(1/1) ...lyplatform/ALL_VALUES_ALL, VertexName: E25B12CAB49E47219630C22B77CE8A45, VertexParallelism: 1, TaskAttemptID:attempt_1452791809534_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BEE8090BA19646B99F5BFEAC84D7DFB7, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=57E9F985B3AD48C3947C2EAC78F5DB71, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:49,762 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791809534_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:49,762 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:49,762 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791809534_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:49,762 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791809534_0001_00_000001 completed successfully
2016-01-14 17:16:49,762 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791809534_0001_00_000002
2016-01-14 17:16:49,762 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791809534_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:49,763 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791809534_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:49,764 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791809534_0001_00_000002
2016-01-14 17:16:49,764 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 17:16:49,766 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791809534_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:49,766 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809534_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791809534_0001_00_000002, launchTime=1452791809766
2016-01-14 17:16:49,766 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:49,767 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791809534_0001_00_000002
2016-01-14 17:16:49,767 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791809534_0001_1_01_000000_0] to container: [container_1452791809534_0001_00_000002]
2016-01-14 17:16:49,767 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791809534_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:49,767 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791809534_0001_00_000002 given task: attempt_1452791809534_0001_1_01_000000_0
2016-01-14 17:16:49,767 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791809534_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791809534_0001_1_01_000000_0
2016-01-14 17:16:49,768 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791809534_0001_1_01_000000_0] started. Is using containerId: [container_1452791809534_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:49,768 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809534_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E25B12CAB49E47219630C22B77CE8A45, taskAttemptId=attempt_1452791809534_0001_1_01_000000_0, startTime=1452791809767, containerId=container_1452791809534_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791809534_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:49,768 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:49,768 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791809534_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:49,768 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791809534_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:49 Starting to run new task attempt: attempt_1452791809534_0001_1_01_000000_0
2016-01-14 17:16:49,768 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791809534_0001_1_01_000000_0
2016-01-14 17:16:49,768 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:49,768 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:49,768 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:49,768 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:49,768 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:49,768 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:49,769 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [917E8AAA53F548CC897F96481F4AA76A/F3092A1E5B4F4130BB3E2185F1003973] every/(1/1) ...lyplatform/ALL_VALUES_ALL, VertexName: E25B12CAB49E47219630C22B77CE8A45, VertexParallelism: 1, TaskAttemptID:attempt_1452791809534_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BEE8090BA19646B99F5BFEAC84D7DFB7, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=57E9F985B3AD48C3947C2EAC78F5DB71, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:49,769 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=768081920
2016-01-14 17:16:49,769 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791809534_0001_1_01_000000_0
2016-01-14 17:16:49,769 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:49,769 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:49,769 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791809534_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:49,769 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:49,769 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=BEE8090BA19646B99F5BFEAC84D7DFB7, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:49,770 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:49,770 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: BEE8090BA19646B99F5BFEAC84D7DFB7
2016-01-14 17:16:49,771 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 691273728, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:49,772 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:49,772 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 691273728, type: INPUT, componentVertexName: BEE8090BA19646B99F5BFEAC84D7DFB7
2016-01-14 17:16:49,772 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: BEE8090BA19646B99F5BFEAC84D7DFB7
2016-01-14 17:16:49,772 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:49,772 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=57E9F985B3AD48C3947C2EAC78F5DB71, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:49,773 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:49,773 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 57E9F985B3AD48C3947C2EAC78F5DB71
2016-01-14 17:16:49,773 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:49,773 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 57E9F985B3AD48C3947C2EAC78F5DB71
2016-01-14 17:16:49,776 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E25B12CAB49E47219630C22B77CE8A45, ordinal: 1
2016-01-14 17:16:49,776 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E25B12CAB49E47219630C22B77CE8A45, mem on start (mb), free: 509, total: 732, max: 732
2016-01-14 17:16:49,776 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:49,776 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:49,776 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:49,782 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:49,782 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 57E9F985B3AD48C3947C2EAC78F5DB71
2016-01-14 17:16:49,782 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:49,782 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:49,782 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:49,783 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:49,783 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:49,783 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:49,783 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 691273728, TotalRequestedScaled: 6.380988258461539E8, TotalJVMHeap: 768081920, TotalAvailable: 514614886, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:49,783 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, BEE8090BA19646B99F5BFEAC84D7DFB7, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=691273728, allocated=514614886
2016-01-14 17:16:49,783 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 57E9F985B3AD48C3947C2EAC78F5DB71, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:49,783 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:49,783 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: BEE8090BA19646B99F5BFEAC84D7DFB7 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:49,783 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: BEE8090BA19646B99F5BFEAC84D7DFB7
2016-01-14 17:16:49,783 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:49,784 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:49,784 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:49,784 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:49,784 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: BEE8090BA19646B99F5BFEAC84D7DFB7 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:49,784 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=691273728, postMergeMem=0, RuntimeTotalAvailable=514614886. Updated to: ShuffleMem=514614886, postMergeMem: 0
2016-01-14 17:16:49,784 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=514614886, maxSingleShuffleLimit=128653720, mergeThreshold=463153376, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:49,784 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:49,785 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: BEE8090BA19646B99F5BFEAC84D7DFB7
2016-01-14 17:16:49,785 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:49,785 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791809534_0001_1_01_000000_0
2016-01-14 17:16:49,786 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791809534_0001_1_00_000000_0_10302, runDuration: 36086, ]
2016-01-14 17:16:49,787 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791809534_0001_1_00_000000_0_10302] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791809787, TimeTaken=0, Rate=0.00 MB/s
2016-01-14 17:16:49,787 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : BEE8090BA19646B99F5BFEAC84D7DFB7
2016-01-14 17:16:49,788 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:49,788 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [BEE8090BA19646B99F5BFEAC84D7DFB7] #51 in 1ms
2016-01-14 17:16:49,788 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [BEE8090BA19646B99F5BFEAC84D7DFB7] #51, status:false, isInterrupted:false
2016-01-14 17:16:49,788 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:49,789 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: C8CC15511B5948CA8C9E395855796812
2016-01-14 17:16:49,789 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:49,789 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_ALL"], id: 57E9F985B3AD48C3947C2EAC78F5DB71
2016-01-14 17:16:49,789 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:49,789 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_ALL"]
2016-01-14 17:16:49,789 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:49,789 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] C8CC15511B5948CA8C9E395855796812, for 1 inputs
2016-01-14 17:16:49,789 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:49,790 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:49,790 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : BEE8090BA19646B99F5BFEAC84D7DFB7
2016-01-14 17:16:49,790 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:49,790 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E25B12CAB49E47219630C22B77CE8A45, all 1 inputs ready in: 00:00:00.001
2016-01-14 17:16:49,791 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@58780dfe; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@31decc12
2016-01-14 17:16:49,791 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:49,793 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(92)) - caught throwable
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:49,795 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:49,795 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:49,795 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791809534_0001_1_01_000000_0
2016-01-14 17:16:49,795 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791809534_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:49,796 INFO  task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1452791809534_0001_1_01_000000_0
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:49,797 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=768081920, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791809790, Process_Duration=6, Process_End_Time=1452791809796, Read_Duration=0, Tuples_Read=1]]
2016-01-14 17:16:49,798 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E25B12CAB49E47219630C22B77CE8A45, mem on close (mb), free: 505, total: 732, max: 732
2016-01-14 17:16:49,798 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809534_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E25B12CAB49E47219630C22B77CE8A45, taskAttemptId=attempt_1452791809534_0001_1_01_000000_0, startTime=1452791809767, finishTime=1452791809798, timeTaken=31, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=768081920, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791809790, Process_Duration=6, Process_End_Time=1452791809796, Read_Duration=0, Tuples_Read=1
2016-01-14 17:16:49,798 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791809534_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-01-14 17:16:49,798 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=E25B12CAB49E47219630C22B77CE8A45, index=1
2016-01-14 17:16:49,798 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:49,798 INFO  impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1452791809534_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-01-14 17:16:49,799 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809534_0001_1][Event:TASK_FINISHED]: vertexName=E25B12CAB49E47219630C22B77CE8A45, taskId=task_1452791809534_0001_1_01_000000, startTime=1452791809767, finishTime=1452791809799, timeTaken=32, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
], counters=Counters: 0
2016-01-14 17:16:49,799 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:49,799 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791809534_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-01-14 17:16:49,799 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:49,799 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791809534_0001_1_01 [E25B12CAB49E47219630C22B77CE8A45] : 1
2016-01-14 17:16:49,799 INFO  impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1452791809534_0001_1_01 [E25B12CAB49E47219630C22B77CE8A45] because task failed: task_1452791809534_0001_1_01_000000
2016-01-14 17:16:49,799 INFO  impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1452791809534_0001_1_01 [E25B12CAB49E47219630C22B77CE8A45] due to trigger: OWN_TASK_FAILURE
2016-01-14 17:16:49,799 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-01-14 17:16:49,799 INFO  impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1452791809534_0001_1_01 [E25B12CAB49E47219630C22B77CE8A45]
2016-01-14 17:16:49,799 INFO  impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=57E9F985B3AD48C3947C2EAC78F5DB71, vertexId=vertex_1452791809534_0001_1_01 [E25B12CAB49E47219630C22B77CE8A45]
2016-01-14 17:16:49,799 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: BEE8090BA19646B99F5BFEAC84D7DFB7
2016-01-14 17:16:49,800 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:49,800 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:49,800 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:49,800 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:49,800 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:49,801 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=E25B12CAB49E47219630C22B77CE8A45, sourceVertex=BEE8090BA19646B99F5BFEAC84D7DFB7
2016-01-14 17:16:49,801 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809534_0001_1][Event:VERTEX_FINISHED]: vertexName=E25B12CAB49E47219630C22B77CE8A45, vertexId=vertex_1452791809534_0001_1_01, initRequestedTime=1452791809693, initedTime=1452791809700, startRequestedTime=1452791809705, startedTime=1452791809705, finishTime=1452791809800, timeTaken=95, status=FAILED, diagnostics=Task failed, taskId=task_1452791809534_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1452791809534_0001_1_01 [E25B12CAB49E47219630C22B77CE8A45] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1452791809767, firstTasksToStart=[ task_1452791809534_0001_1_01_000000 ], lastTaskFinishTime=1452791809798, lastTasksToFinish=[ task_1452791809534_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[  ], longestDurationTasks=[  ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-01-14 17:16:49,801 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791809534_0001_1_01 [E25B12CAB49E47219630C22B77CE8A45] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-01-14 17:16:49,801 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809534_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791809534_0001_00_000002, stoppedTime=1452791809801, exitStatus=0
2016-01-14 17:16:49,801 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:49,801 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791809534_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:49,802 INFO  node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1452791809534_0001_1_01_000000_0 failed: true container: container_1452791809534_0001_00_000002 numFailedTAs: 0
2016-01-14 17:16:49,802 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791809534_0001_1_01 [E25B12CAB49E47219630C22B77CE8A45] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-01-14 17:16:49,802 INFO  impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-01-14 17:16:49,802 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791809534_0001_1
2016-01-14 17:16:49,802 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1452791809534_0001_1_00 [BEE8090BA19646B99F5BFEAC84D7DFB7]
2016-01-14 17:16:49,802 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 57E9F985B3AD48C3947C2EAC78F5DB71 for vertex: vertex_1452791809534_0001_1_01 [E25B12CAB49E47219630C22B77CE8A45]
2016-01-14 17:16:49,801 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=E25B12CAB49E47219630C22B77CE8A45, sourceVertex=57E9F985B3AD48C3947C2EAC78F5DB71
2016-01-14 17:16:49,802 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:49,803 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791809534_0001_00_000002
2016-01-14 17:16:49,803 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791809534_0001_00_000002
2016-01-14 17:16:49,803 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791809534_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-01-14 17:16:49,803 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1452791809534_0001_00_000002 completed but with errors
2016-01-14 17:16:49,804 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791809534_0001_1, queueSize=0
2016-01-14 17:16:49,804 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:49,804 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809534_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791809534_0001_1, startTime=1452791809691, finishTime=1452791809803, timeTaken=112, status=FAILED, diagnostics=Vertex failed, vertexName=E25B12CAB49E47219630C22B77CE8A45, vertexId=vertex_1452791809534_0001_1_01, diagnostics=[Task failed, taskId=task_1452791809534_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1452791809534_0001_1_01 [E25B12CAB49E47219630C22B77CE8A45] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=768081920, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791809744, Process_Duration=3, Process_End_Time=1452791809747, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:49,804 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791809534_0001_1 finished with state: FAILED
2016-01-14 17:16:49,804 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791809534_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:49,805 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452791809534_0001_00_000002
2016-01-14 17:16:49,805 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1452791809534_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-01-14 17:16:49,805 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791809534_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:16:49 Completed Dag: dag_1452791809534_0001_1
2016-01-14 17:16:49,805 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791809534_0001_1, dagState=FAILED
2016-01-14 17:16:49,806 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791809534_0001_1_post
2016-01-14 17:16:49,806 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:49,806 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:49,806 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791809534_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-01-14 17:16:49,808 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...lyplatform/ALL_VALUES_ALL, applicationId=application_1452791809534_0001
2016-01-14 17:16:49,810 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:49,810 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:49,810 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:49,810 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:49,810 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:49,811 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:49,811 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:49,811 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:49,812 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:49,812 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:49,812 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:49,812 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:49,812 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:49,818 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/917E8AAA53F548CC897F96481F4AA76A/F3092A1E5B4F4130BB3E2185F1003973/.tez/application_1452791809534_0001
2016-01-14 17:16:49,818 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping all jobs
2016-01-14 17:16:49,818 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping: (1/1) ...lyplatform/ALL_VALUES_ALL
2016-01-14 17:16:49,819 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopped all jobs
2016-01-14 17:16:49,819 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.302
2016-01-14 17:16:49,819 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_VALUES_ALL
2016-01-14 17:16:49,839 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:49,839 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:49,874 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-01-14 17:16:49,875 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-01-14 17:16:49,876 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:49,876 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:49,876 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:49,898 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:49,902 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_REPLACE
2016-01-14 17:16:49,943 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:49,943 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:49,944 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_REPLACE"]
2016-01-14 17:16:49,944 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:49,944 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:49,944 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:49,944 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...atform/ALL_VALUES_REPLACE
2016-01-14 17:16:49,944 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:49,955 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:49,960 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:49,960 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:49,960 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:49,969 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A4F8D1FB364444CFB614B9944D64BB10/EBE900D6BBAC4DF199670B2089098729/.tez/application_1452791809960_0001 doesn't exist and is created
2016-01-14 17:16:49,985 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:49,985 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:49,986 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A4F8D1FB364444CFB614B9944D64BB10/EBE900D6BBAC4DF199670B2089098729/.tez/application_1452791809960_0001
2016-01-14 17:16:49,988 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791809960_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:49,998 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:49,999 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:50,000 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:50,000 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:50,000 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A4F8D1FB364444CFB614B9944D64BB10/EBE900D6BBAC4DF199670B2089098729/.tez/application_1452791809960_0001/localmode-log-dir
2016-01-14 17:16:50,001 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A4F8D1FB364444CFB614B9944D64BB10/EBE900D6BBAC4DF199670B2089098729/.tez/application_1452791809960_0001/localmode-log-dir/history.txt.appattempt_1452791809960_0001_000000, maxErrors=10
2016-01-14 17:16:50,001 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:50,001 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791809960_0001_000000, appSubmitTime=1452791809988, launchTime=1452791809988
2016-01-14 17:16:50,005 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:50,005 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:50,006 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:47490
2016-01-14 17:16:50,006 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:50,007 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:50,009 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:50,009 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791809960_0001_000000, startTime=1452791810009
2016-01-14 17:16:50,009 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:50,010 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:50,080 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 17:16:50,080 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 17:16:50,085 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:50,085 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:50,085 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...atform/ALL_VALUES_REPLACE, applicationId=application_1452791809960_0001, dagName=[A4F8D1FB364444CFB614B9944D64BB10/EBE900D6BBAC4DF199670B2089098729] every/(1/1) ...atform/ALL_VALUES_REPLACE
2016-01-14 17:16:50,100 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [A4F8D1FB364444CFB614B9944D64BB10/EBE900D6BBAC4DF199670B2089098729] every/(1/1) ...atform/ALL_VALUES_REPLACE
2016-01-14 17:16:50,101 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791809960_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A4F8D1FB364444CFB614B9944D64BB10/EBE900D6BBAC4DF199670B2089098729/.tez/application_1452791809960_0001/localmode-log-dir/dag_1452791809960_0001_1.dot
2016-01-14 17:16:50,102 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A4F8D1FB364444CFB614B9944D64BB10/EBE900D6BBAC4DF199670B2089098729/.tez/application_1452791809960_0001/localmode-log-dir/dag_1452791809960_0001_1-tez-dag.pb.txt
2016-01-14 17:16:50,107 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791809960_0001_1
2016-01-14 17:16:50,107 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:50,108 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [A4F8D1FB364444CFB614B9944D64BB10/EBE900D6BBAC4DF199670B2089098729] every/(1/1) ...atform/ALL_VALUES_REPLACE
2016-01-14 17:16:50 Running Dag: dag_1452791809960_0001_1
2016-01-14 17:16:50,120 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:50,120 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809960_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791809960_0001_1, submitTime=1452791810100
2016-01-14 17:16:50,121 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex BBCEEAA7B49F4977815FE0073FD3239C
2016-01-14 17:16:50,121 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:50,121 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:50,122 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809960_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791809960_0001_1, initTime=1452791810120
2016-01-14 17:16:50,122 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791809960_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:50,122 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809960_0001_1][Event:DAG_STARTED]: dagID=dag_1452791809960_0001_1, startTime=1452791810122
2016-01-14 17:16:50,122 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:50,122 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791809960_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:50,123 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: A40782169C664C4B9A04D0C392A4E5BD : {4F58A62442B24258B5C06CFA8DBBBC67={InputName=4F58A62442B24258B5C06CFA8DBBBC67}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:50,123 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 4F58A62442B24258B5C06CFA8DBBBC67, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:50,123 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791809960_0001_1_00 [A40782169C664C4B9A04D0C392A4E5BD]
2016-01-14 17:16:50,123 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791809960_0001_1_00 [A40782169C664C4B9A04D0C392A4E5BD]
2016-01-14 17:16:50,123 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791809960_0001_1_00 [A40782169C664C4B9A04D0C392A4E5BD]
2016-01-14 17:16:50,123 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791809960_0001_1_00 [A40782169C664C4B9A04D0C392A4E5BD]. Starting root input initializers: 1
2016-01-14 17:16:50,122 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...atform/ALL_VALUES_REPLACE, applicationId=application_1452791809960_0001, dagName=[A4F8D1FB364444CFB614B9944D64BB10/EBE900D6BBAC4DF199670B2089098729] every/(1/1) ...atform/ALL_VALUES_REPLACE
2016-01-14 17:16:50,122 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:50,124 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791809960_0001_1_00 [A40782169C664C4B9A04D0C392A4E5BD] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:50,124 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791809960_0001_1_01 [BBCEEAA7B49F4977815FE0073FD3239C]
2016-01-14 17:16:50,125 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 4F58A62442B24258B5C06CFA8DBBBC67 on vertex vertex_1452791809960_0001_1_00 [A40782169C664C4B9A04D0C392A4E5BD]
2016-01-14 17:16:50,126 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:50,126 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 4F58A62442B24258B5C06CFA8DBBBC67 asking for 1 tasks. Headroom: 731 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:50,127 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:50,127 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:50,127 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:50,127 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791809960_0001_1_01 [BBCEEAA7B49F4977815FE0073FD3239C]
2016-01-14 17:16:50,127 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791809960_0001_1_01 [BBCEEAA7B49F4977815FE0073FD3239C]
2016-01-14 17:16:50,127 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791809960_0001_1_01 [BBCEEAA7B49F4977815FE0073FD3239C]
2016-01-14 17:16:50,128 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=2A9FDB5DAAE54BAB9956AE7BB245B754, vertexId=vertex_1452791809960_0001_1_01 [BBCEEAA7B49F4977815FE0073FD3239C], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:50,128 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:50,129 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 4F58A62442B24258B5C06CFA8DBBBC67 on vertex vertex_1452791809960_0001_1_00 [A40782169C664C4B9A04D0C392A4E5BD]
2016-01-14 17:16:50,129 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=2A9FDB5DAAE54BAB9956AE7BB245B754, vertexId=vertex_1452791809960_0001_1_01 [BBCEEAA7B49F4977815FE0073FD3239C]
2016-01-14 17:16:50,131 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=2A9FDB5DAAE54BAB9956AE7BB245B754, vertexName=BBCEEAA7B49F4977815FE0073FD3239C, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:50,131 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=2A9FDB5DAAE54BAB9956AE7BB245B754, vertexId=vertex_1452791809960_0001_1_01 [BBCEEAA7B49F4977815FE0073FD3239C]
2016-01-14 17:16:50,133 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:50,133 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809960_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BBCEEAA7B49F4977815FE0073FD3239C, vertexId=vertex_1452791809960_0001_1_01, initRequestedTime=1452791810124, initedTime=1452791810133, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:50,133 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791809960_0001_1_01 [BBCEEAA7B49F4977815FE0073FD3239C] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:50,133 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791809960_0001_1_00 [A40782169C664C4B9A04D0C392A4E5BD] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:50,133 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {4F58A62442B24258B5C06CFA8DBBBC67=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:50,134 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791809960_0001_1_00 [A40782169C664C4B9A04D0C392A4E5BD] parallelism set to 1
2016-01-14 17:16:50,134 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791809960_0001_1_00 [A40782169C664C4B9A04D0C392A4E5BD] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:50,134 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809960_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791809960_0001_1_00, eventCount=1
2016-01-14 17:16:50,134 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809960_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A40782169C664C4B9A04D0C392A4E5BD, vertexId=vertex_1452791809960_0001_1_00, initRequestedTime=1452791810123, initedTime=1452791810134, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:50,135 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791809960_0001_1_00 [A40782169C664C4B9A04D0C392A4E5BD] with distanceFromRoot: 0
2016-01-14 17:16:50,134 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:50,135 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791809960_0001_1_00 [A40782169C664C4B9A04D0C392A4E5BD] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:50,135 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in A40782169C664C4B9A04D0C392A4E5BD
2016-01-14 17:16:50,136 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791809960_0001_1_00 [A40782169C664C4B9A04D0C392A4E5BD]
2016-01-14 17:16:50,136 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809960_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791809960_0001_1_00, startRequestedTime=1452791810135, startedTime=1452791810135
2016-01-14 17:16:50,136 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:50,136 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791809960_0001_1_00 [A40782169C664C4B9A04D0C392A4E5BD] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:50,137 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:50,137 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809960_0001_1][Event:TASK_STARTED]: vertexName=A40782169C664C4B9A04D0C392A4E5BD, taskId=task_1452791809960_0001_1_00_000000, scheduledTime=1452791810136, launchTime=1452791810136
2016-01-14 17:16:50,137 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791809960_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:50,137 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791809960_0001_1_00 for vertex: vertex_1452791809960_0001_1_01 [BBCEEAA7B49F4977815FE0073FD3239C] numStartedSources: 1 numSources: 1
2016-01-14 17:16:50,137 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791809960_0001_1_01 [BBCEEAA7B49F4977815FE0073FD3239C] with distanceFromRoot: 1
2016-01-14 17:16:50,138 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791809960_0001_1_00 [A40782169C664C4B9A04D0C392A4E5BD]
2016-01-14 17:16:50,138 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: A40782169C664C4B9A04D0C392A4E5BD in vertex: BBCEEAA7B49F4977815FE0073FD3239C
2016-01-14 17:16:50,138 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: BBCEEAA7B49F4977815FE0073FD3239C with 1 source tasks and 1 pending tasks
2016-01-14 17:16:50,138 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809960_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791809960_0001_1_01, startRequestedTime=1452791810138, startedTime=1452791810138
2016-01-14 17:16:50,138 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:50,139 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791809960_0001_1_01 [BBCEEAA7B49F4977815FE0073FD3239C] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:50,139 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [A4F8D1FB364444CFB614B9944D64BB10/EBE900D6BBAC4DF199670B2089098729] every/(1/1) ...atform/ALL_VALUES_REPLACE, VertexName: A40782169C664C4B9A04D0C392A4E5BD, VertexParallelism: 1, TaskAttemptID:attempt_1452791809960_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=4F58A62442B24258B5C06CFA8DBBBC67, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=BBCEEAA7B49F4977815FE0073FD3239C, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:50,139 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791809960_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:50,139 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:50,140 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:50,140 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:50,141 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:50,141 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791809960_0001_00_000001
2016-01-14 17:16:50,141 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791809960_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:50,141 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791809960_0001, with dag id: dag_1452791809960_0001_1
2016-01-14 17:16:50,143 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791809960_0001_00_000001
2016-01-14 17:16:50,143 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:50,143 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791809960_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:50,143 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:50,143 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809960_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791809960_0001_00_000001, launchTime=1452791810143
2016-01-14 17:16:50,144 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791809960_0001_00_000001
2016-01-14 17:16:50,145 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791809960_0001_1_00_000000_0] to container: [container_1452791809960_0001_00_000001]
2016-01-14 17:16:50,145 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791809960_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:50,145 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791809960_0001_00_000001 given task: attempt_1452791809960_0001_1_00_000000_0
2016-01-14 17:16:50,146 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791809960_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791809960_0001_1_00_000000_0
2016-01-14 17:16:50,146 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791809960_0001_1_00_000000_0] started. Is using containerId: [container_1452791809960_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:50,146 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:50,146 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809960_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A40782169C664C4B9A04D0C392A4E5BD, taskAttemptId=attempt_1452791809960_0001_1_00_000000_0, startTime=1452791810145, containerId=container_1452791809960_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791809960_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:50,146 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791809960_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:50,146 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791809960_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:50 Starting to run new task attempt: attempt_1452791809960_0001_1_00_000000_0
2016-01-14 17:16:50,147 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791809960_0001_1_00_000000_0
2016-01-14 17:16:50,147 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:50,148 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:50,148 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:50,148 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:50,148 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:50,148 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:50,148 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [A4F8D1FB364444CFB614B9944D64BB10/EBE900D6BBAC4DF199670B2089098729] every/(1/1) ...atform/ALL_VALUES_REPLACE, VertexName: A40782169C664C4B9A04D0C392A4E5BD, VertexParallelism: 1, TaskAttemptID:attempt_1452791809960_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=4F58A62442B24258B5C06CFA8DBBBC67, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=BBCEEAA7B49F4977815FE0073FD3239C, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:50,148 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=767033344
2016-01-14 17:16:50,149 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791809960_0001_1_00_000000_0
2016-01-14 17:16:50,149 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:50,149 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:50,149 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791809960_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:50,150 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=4F58A62442B24258B5C06CFA8DBBBC67, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:50,150 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=BBCEEAA7B49F4977815FE0073FD3239C, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:50,150 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:50,150 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:50,150 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 4F58A62442B24258B5C06CFA8DBBBC67
2016-01-14 17:16:50,150 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 4F58A62442B24258B5C06CFA8DBBBC67
2016-01-14 17:16:50,151 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:50,151 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: BBCEEAA7B49F4977815FE0073FD3239C
2016-01-14 17:16:50,152 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:50,152 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 4F58A62442B24258B5C06CFA8DBBBC67
2016-01-14 17:16:50,152 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 4F58A62442B24258B5C06CFA8DBBBC67
2016-01-14 17:16:50,153 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:50,153 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:50,154 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: BBCEEAA7B49F4977815FE0073FD3239C
2016-01-14 17:16:50,154 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: BBCEEAA7B49F4977815FE0073FD3239C
2016-01-14 17:16:50,155 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A40782169C664C4B9A04D0C392A4E5BD, ordinal: 0
2016-01-14 17:16:50,155 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A40782169C664C4B9A04D0C392A4E5BD, mem on start (mb), free: 623, total: 731, max: 731
2016-01-14 17:16:50,155 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:50,155 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:50,155 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:50,156 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:50,156 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:50,156 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:50,156 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:50,156 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:50,156 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:50,156 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 767033344, TotalAvailable: 513912340, TotalRequested/TotalJVMHeap:0.14
2016-01-14 17:16:50,156 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 4F58A62442B24258B5C06CFA8DBBBC67, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:50,156 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, BBCEEAA7B49F4977815FE0073FD3239C, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:50,157 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:50,157 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 4F58A62442B24258B5C06CFA8DBBBC67 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:50,157 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 4F58A62442B24258B5C06CFA8DBBBC67
2016-01-14 17:16:50,157 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:50,157 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 4F58A62442B24258B5C06CFA8DBBBC67
2016-01-14 17:16:50,157 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:50,157 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:50,157 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:50,157 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791809960_0001_1_00_000000_0
2016-01-14 17:16:50,158 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:50,158 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@6b35255a
2016-01-14 17:16:50,159 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@6b35255a
2016-01-14 17:16:50,159 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:50,159 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:50,162 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:50,162 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 4F58A62442B24258B5C06CFA8DBBBC67
2016-01-14 17:16:50,162 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: B108B7E2FD2E4DDC932F27F9D6E61DD6
2016-01-14 17:16:50,162 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] B108B7E2FD2E4DDC932F27F9D6E61DD6
2016-01-14 17:16:50,163 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@3b6e6c7e; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@4e94e2ad; comparator=cascading.tuple.hadoop.util.TupleComparator@10adf429; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:50,163 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:50,163 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:50,163 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:50,177 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:50,177 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:50,177 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:50,177 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:50,177 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:50,178 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:50,178 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:50,178 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A40782169C664C4B9A04D0C392A4E5BD, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:50,180 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791809960_0001_1_00_000000_0
2016-01-14 17:16:50,180 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A40782169C664C4B9A04D0C392A4E5BD, mem on close (mb), free: 521, total: 731, max: 731
2016-01-14 17:16:50,180 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:50,180 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:50,180 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:50,181 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:50,183 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:50 Completed running task attempt: attempt_1452791809960_0001_1_00_000000_0
2016-01-14 17:16:50,186 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791809960_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:50,186 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809960_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791809960_0001_1_00, eventCount=1
2016-01-14 17:16:50,187 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:50,187 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809960_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A40782169C664C4B9A04D0C392A4E5BD, taskAttemptId=attempt_1452791809960_0001_1_00_000000_0, startTime=1452791810145, finishTime=1452791810187, timeTaken=42, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=767033344, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791810178, Process_Duration=2, Process_End_Time=1452791810180, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:50,187 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791809960_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:50,187 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791809960_0001_1_00_000000_0
2016-01-14 17:16:50,187 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:50,187 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:50,187 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809960_0001_1][Event:TASK_FINISHED]: vertexName=A40782169C664C4B9A04D0C392A4E5BD, taskId=task_1452791809960_0001_1_00_000000, startTime=1452791810145, finishTime=1452791810187, timeTaken=42, status=SUCCEEDED, successfulAttemptID=attempt_1452791809960_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=767033344, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791810178, Process_Duration=2, Process_End_Time=1452791810180, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:50,188 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791809960_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:50,188 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791809960_0001_1_00 [A40782169C664C4B9A04D0C392A4E5BD] : 1
2016-01-14 17:16:50,188 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791809960_0001_1_00 [A40782169C664C4B9A04D0C392A4E5BD]
2016-01-14 17:16:50,188 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:50,188 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809960_0001_1][Event:VERTEX_FINISHED]: vertexName=A40782169C664C4B9A04D0C392A4E5BD, vertexId=vertex_1452791809960_0001_1_00, initRequestedTime=1452791810123, initedTime=1452791810134, startRequestedTime=1452791810135, startedTime=1452791810135, finishTime=1452791810188, timeTaken=53, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=767033344, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791810178, Process_Duration=2, Process_End_Time=1452791810180, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791810145, firstTasksToStart=[ task_1452791809960_0001_1_00_000000 ], lastTaskFinishTime=1452791810187, lastTasksToFinish=[ task_1452791809960_0001_1_00_000000 ], minTaskDuration=42, maxTaskDuration=42, avgTaskDuration=42.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791809960_0001_1_00_000000 ], longestDurationTasks=[ task_1452791809960_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:50,189 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791809960_0001_1_00 [A40782169C664C4B9A04D0C392A4E5BD] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:50,189 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791809960_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:50,189 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809960_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791809960_0001_00_000001, stoppedTime=1452791810189, exitStatus=0
2016-01-14 17:16:50,189 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:50,189 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791809960_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:50,190 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791809960_0001_1_01 [BBCEEAA7B49F4977815FE0073FD3239C] attempt: attempt_1452791809960_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:50,190 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: BBCEEAA7B49F4977815FE0073FD3239C
2016-01-14 17:16:50,190 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791809960_0001_1_01 [BBCEEAA7B49F4977815FE0073FD3239C]
2016-01-14 17:16:50,190 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791809960_0001_1_00 [A40782169C664C4B9A04D0C392A4E5BD] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:50,190 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791809960_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:50,191 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809960_0001_1][Event:TASK_STARTED]: vertexName=BBCEEAA7B49F4977815FE0073FD3239C, taskId=task_1452791809960_0001_1_01_000000, scheduledTime=1452791810191, launchTime=1452791810191
2016-01-14 17:16:50,191 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:50,191 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791809960_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:50,191 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791809960_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:50,192 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791809960_0001_1_01 [BBCEEAA7B49F4977815FE0073FD3239C]
2016-01-14 17:16:50,192 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [A4F8D1FB364444CFB614B9944D64BB10/EBE900D6BBAC4DF199670B2089098729] every/(1/1) ...atform/ALL_VALUES_REPLACE, VertexName: BBCEEAA7B49F4977815FE0073FD3239C, VertexParallelism: 1, TaskAttemptID:attempt_1452791809960_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A40782169C664C4B9A04D0C392A4E5BD, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=2A9FDB5DAAE54BAB9956AE7BB245B754, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:50,192 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791809960_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:50,192 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:50,192 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=767033344, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791810178, Process_Duration=2, Process_End_Time=1452791810180, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:50,193 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791809960_0001_00_000002
2016-01-14 17:16:50,193 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:50,194 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:50,193 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791809960_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:50,194 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791809960_0001_00_000001
2016-01-14 17:16:50,194 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791809960_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:50,194 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791809960_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:50,195 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791809960_0001_00_000001, Exiting
2016-01-14 17:16:50,195 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791809960_0001_00_000001
2016-01-14 17:16:50,195 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791809960_0001_00_000001
2016-01-14 17:16:50,195 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791809960_0001_00_000002
2016-01-14 17:16:50,196 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791809960_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:50,196 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 17:16:50,196 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791809960_0001_00_000001 completed successfully
2016-01-14 17:16:50,198 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791809960_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:50,198 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:50,199 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809960_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791809960_0001_00_000002, launchTime=1452791810198
2016-01-14 17:16:50,198 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791809960_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:50,199 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791809960_0001_00_000002
2016-01-14 17:16:50,200 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791809960_0001_1_01_000000_0] to container: [container_1452791809960_0001_00_000002]
2016-01-14 17:16:50,200 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791809960_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:50,200 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791809960_0001_00_000002 given task: attempt_1452791809960_0001_1_01_000000_0
2016-01-14 17:16:50,200 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791809960_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791809960_0001_1_01_000000_0
2016-01-14 17:16:50,200 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791809960_0001_1_01_000000_0] started. Is using containerId: [container_1452791809960_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:50,200 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:50,200 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809960_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BBCEEAA7B49F4977815FE0073FD3239C, taskAttemptId=attempt_1452791809960_0001_1_01_000000_0, startTime=1452791810200, containerId=container_1452791809960_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791809960_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:50,201 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791809960_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:50,201 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791809960_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:50 Starting to run new task attempt: attempt_1452791809960_0001_1_01_000000_0
2016-01-14 17:16:50,201 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791809960_0001_1_01_000000_0
2016-01-14 17:16:50,201 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:50,201 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:50,201 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:50,202 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:50,202 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:50,202 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:50,202 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [A4F8D1FB364444CFB614B9944D64BB10/EBE900D6BBAC4DF199670B2089098729] every/(1/1) ...atform/ALL_VALUES_REPLACE, VertexName: BBCEEAA7B49F4977815FE0073FD3239C, VertexParallelism: 1, TaskAttemptID:attempt_1452791809960_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A40782169C664C4B9A04D0C392A4E5BD, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=2A9FDB5DAAE54BAB9956AE7BB245B754, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:50,202 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=767033344
2016-01-14 17:16:50,203 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791809960_0001_1_01_000000_0
2016-01-14 17:16:50,203 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:50,203 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:50,203 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791809960_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:50,204 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:50,204 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=A40782169C664C4B9A04D0C392A4E5BD, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:50,204 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=2A9FDB5DAAE54BAB9956AE7BB245B754, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:50,204 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:50,204 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: A40782169C664C4B9A04D0C392A4E5BD
2016-01-14 17:16:50,205 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:50,205 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 2A9FDB5DAAE54BAB9956AE7BB245B754
2016-01-14 17:16:50,205 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:50,205 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 2A9FDB5DAAE54BAB9956AE7BB245B754
2016-01-14 17:16:50,206 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 690329984, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:50,206 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:50,206 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 690329984, type: INPUT, componentVertexName: A40782169C664C4B9A04D0C392A4E5BD
2016-01-14 17:16:50,206 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: A40782169C664C4B9A04D0C392A4E5BD
2016-01-14 17:16:50,207 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:50,212 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BBCEEAA7B49F4977815FE0073FD3239C, ordinal: 1
2016-01-14 17:16:50,212 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BBCEEAA7B49F4977815FE0073FD3239C, mem on start (mb), free: 515, total: 731, max: 731
2016-01-14 17:16:50,212 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:50,212 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:50,212 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:50,218 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:50,218 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 2A9FDB5DAAE54BAB9956AE7BB245B754
2016-01-14 17:16:50,218 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:50,218 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:50,218 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:50,218 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:50,218 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:50,218 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:50,218 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 690329984, TotalRequestedScaled: 6.372276775384616E8, TotalJVMHeap: 767033344, TotalAvailable: 513912340, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:50,218 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 2A9FDB5DAAE54BAB9956AE7BB245B754, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:50,219 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, A40782169C664C4B9A04D0C392A4E5BD, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=690329984, allocated=513912340
2016-01-14 17:16:50,219 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:50,219 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: A40782169C664C4B9A04D0C392A4E5BD being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:50,219 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:50,219 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:50,219 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: A40782169C664C4B9A04D0C392A4E5BD
2016-01-14 17:16:50,220 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:50,220 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:50,221 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: A40782169C664C4B9A04D0C392A4E5BD with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:50,221 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=690329984, postMergeMem=0, RuntimeTotalAvailable=513912340. Updated to: ShuffleMem=513912340, postMergeMem: 0
2016-01-14 17:16:50,221 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=513912340, maxSingleShuffleLimit=128478088, mergeThreshold=462521120, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:50,221 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:50,222 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: A40782169C664C4B9A04D0C392A4E5BD
2016-01-14 17:16:50,222 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:50,222 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791809960_0001_1_01_000000_0
2016-01-14 17:16:50,225 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:50,225 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: B108B7E2FD2E4DDC932F27F9D6E61DD6
2016-01-14 17:16:50,225 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_REPLACE"], id: 2A9FDB5DAAE54BAB9956AE7BB245B754
2016-01-14 17:16:50,225 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_REPLACE"]
2016-01-14 17:16:50,226 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] B108B7E2FD2E4DDC932F27F9D6E61DD6, for 1 inputs
2016-01-14 17:16:50,226 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791809960_0001_1_00_000000_0_10309, runDuration: 32992, ]
2016-01-14 17:16:50,227 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791809960_0001_1_00_000000_0_10309] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791810227, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 17:16:50,227 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : A40782169C664C4B9A04D0C392A4E5BD
2016-01-14 17:16:50,227 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:50,228 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [A40782169C664C4B9A04D0C392A4E5BD] #52 in 2ms
2016-01-14 17:16:50,228 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [A40782169C664C4B9A04D0C392A4E5BD] #52, status:false, isInterrupted:false
2016-01-14 17:16:50,228 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:50,228 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:50,228 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:50,229 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:50,229 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:50,229 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : A40782169C664C4B9A04D0C392A4E5BD
2016-01-14 17:16:50,229 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:50,229 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BBCEEAA7B49F4977815FE0073FD3239C, all 1 inputs ready in: 00:00:00.003
2016-01-14 17:16:50,229 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@332f6e67; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@6ef94671
2016-01-14 17:16:50,230 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:50,230 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(92)) - caught throwable
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:50,230 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:50,230 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:50,231 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791809960_0001_1_01_000000_0
2016-01-14 17:16:50,231 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791809960_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:50,232 INFO  task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1452791809960_0001_1_01_000000_0
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:50,233 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=767033344, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791810229, Process_Duration=3, Process_End_Time=1452791810232, Read_Duration=1, Tuples_Read=1]]
2016-01-14 17:16:50,234 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:50,234 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809960_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BBCEEAA7B49F4977815FE0073FD3239C, taskAttemptId=attempt_1452791809960_0001_1_01_000000_0, startTime=1452791810200, finishTime=1452791810233, timeTaken=33, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=767033344, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791810229, Process_Duration=3, Process_End_Time=1452791810232, Read_Duration=1, Tuples_Read=1
2016-01-14 17:16:50,234 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791809960_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-01-14 17:16:50,235 INFO  impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1452791809960_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-01-14 17:16:50,235 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:50,234 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BBCEEAA7B49F4977815FE0073FD3239C, mem on close (mb), free: 510, total: 731, max: 731
2016-01-14 17:16:50,235 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809960_0001_1][Event:TASK_FINISHED]: vertexName=BBCEEAA7B49F4977815FE0073FD3239C, taskId=task_1452791809960_0001_1_01_000000, startTime=1452791810200, finishTime=1452791810235, timeTaken=35, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
], counters=Counters: 0
2016-01-14 17:16:50,235 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791809960_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-01-14 17:16:50,236 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:50,236 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=BBCEEAA7B49F4977815FE0073FD3239C, index=1
2016-01-14 17:16:50,236 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: A40782169C664C4B9A04D0C392A4E5BD
2016-01-14 17:16:50,236 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:50,236 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791809960_0001_1_01 [BBCEEAA7B49F4977815FE0073FD3239C] : 1
2016-01-14 17:16:50,237 INFO  impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1452791809960_0001_1_01 [BBCEEAA7B49F4977815FE0073FD3239C] because task failed: task_1452791809960_0001_1_01_000000
2016-01-14 17:16:50,237 INFO  impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1452791809960_0001_1_01 [BBCEEAA7B49F4977815FE0073FD3239C] due to trigger: OWN_TASK_FAILURE
2016-01-14 17:16:50,237 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-01-14 17:16:50,237 INFO  impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1452791809960_0001_1_01 [BBCEEAA7B49F4977815FE0073FD3239C]
2016-01-14 17:16:50,237 INFO  impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=2A9FDB5DAAE54BAB9956AE7BB245B754, vertexId=vertex_1452791809960_0001_1_01 [BBCEEAA7B49F4977815FE0073FD3239C]
2016-01-14 17:16:50,238 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:50,238 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:50,238 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809960_0001_1][Event:VERTEX_FINISHED]: vertexName=BBCEEAA7B49F4977815FE0073FD3239C, vertexId=vertex_1452791809960_0001_1_01, initRequestedTime=1452791810124, initedTime=1452791810133, startRequestedTime=1452791810138, startedTime=1452791810138, finishTime=1452791810238, timeTaken=100, status=FAILED, diagnostics=Task failed, taskId=task_1452791809960_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1452791809960_0001_1_01 [BBCEEAA7B49F4977815FE0073FD3239C] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1452791810200, firstTasksToStart=[ task_1452791809960_0001_1_01_000000 ], lastTaskFinishTime=1452791810233, lastTasksToFinish=[ task_1452791809960_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[  ], longestDurationTasks=[  ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-01-14 17:16:50,238 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791809960_0001_1_01 [BBCEEAA7B49F4977815FE0073FD3239C] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-01-14 17:16:50,238 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809960_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791809960_0001_00_000002, stoppedTime=1452791810238, exitStatus=0
2016-01-14 17:16:50,239 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791809960_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:50,239 INFO  node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1452791809960_0001_1_01_000000_0 failed: true container: container_1452791809960_0001_00_000002 numFailedTAs: 0
2016-01-14 17:16:50,239 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791809960_0001_1_01 [BBCEEAA7B49F4977815FE0073FD3239C] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-01-14 17:16:50,239 INFO  impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-01-14 17:16:50,239 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791809960_0001_1
2016-01-14 17:16:50,239 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1452791809960_0001_1_00 [A40782169C664C4B9A04D0C392A4E5BD]
2016-01-14 17:16:50,239 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 2A9FDB5DAAE54BAB9956AE7BB245B754 for vertex: vertex_1452791809960_0001_1_01 [BBCEEAA7B49F4977815FE0073FD3239C]
2016-01-14 17:16:50,238 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:50,238 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:50,240 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:50,240 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:50,240 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=BBCEEAA7B49F4977815FE0073FD3239C, sourceVertex=A40782169C664C4B9A04D0C392A4E5BD
2016-01-14 17:16:50,241 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=BBCEEAA7B49F4977815FE0073FD3239C, sourceVertex=2A9FDB5DAAE54BAB9956AE7BB245B754
2016-01-14 17:16:50,241 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791809960_0001_00_000002
2016-01-14 17:16:50,241 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791809960_0001_00_000002
2016-01-14 17:16:50,241 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791809960_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-01-14 17:16:50,241 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1452791809960_0001_00_000002 completed but with errors
2016-01-14 17:16:50,242 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791809960_0001_1, queueSize=0
2016-01-14 17:16:50,243 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:50,243 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791809960_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791809960_0001_1, startTime=1452791810122, finishTime=1452791810239, timeTaken=117, status=FAILED, diagnostics=Vertex failed, vertexName=BBCEEAA7B49F4977815FE0073FD3239C, vertexId=vertex_1452791809960_0001_1_01, diagnostics=[Task failed, taskId=task_1452791809960_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1452791809960_0001_1_01 [BBCEEAA7B49F4977815FE0073FD3239C] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=767033344, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791810178, Process_Duration=2, Process_End_Time=1452791810180, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:50,243 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791809960_0001_1 finished with state: FAILED
2016-01-14 17:16:50,243 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791809960_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:50,243 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452791809960_0001_00_000002
2016-01-14 17:16:50,243 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1452791809960_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-01-14 17:16:50,243 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791809960_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:16:50 Completed Dag: dag_1452791809960_0001_1
2016-01-14 17:16:50,244 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791809960_0001_1, dagState=FAILED
2016-01-14 17:16:50,244 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791809960_0001_1_post
2016-01-14 17:16:50,244 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:50,244 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:50,244 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791809960_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-01-14 17:16:50,254 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...atform/ALL_VALUES_REPLACE, applicationId=application_1452791809960_0001
2016-01-14 17:16:50,255 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:50,255 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:50,255 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:50,255 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:50,255 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:50,256 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:50,256 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:50,256 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:50,256 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:50,256 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:50,256 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:50,257 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:50,257 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:50,262 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A4F8D1FB364444CFB614B9944D64BB10/EBE900D6BBAC4DF199670B2089098729/.tez/application_1452791809960_0001
2016-01-14 17:16:50,262 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping all jobs
2016-01-14 17:16:50,263 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping: (1/1) ...atform/ALL_VALUES_REPLACE
2016-01-14 17:16:50,263 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopped all jobs
2016-01-14 17:16:50,263 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.320
2016-01-14 17:16:50,263 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_VALUES_REPLACE
2016-01-14 17:16:50,283 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:50,284 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:50,311 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.005
2016-01-14 17:16:50,316 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-01-14 17:16:50,316 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:50,316 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:50,316 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:50,333 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:50,337 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_SWAP
2016-01-14 17:16:50,399 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:50,399 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:50,399 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_SWAP"]
2016-01-14 17:16:50,399 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:50,399 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:50,399 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:50,400 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...yplatform/ALL_VALUES_SWAP
2016-01-14 17:16:50,400 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:50,412 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:50,416 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:50,416 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:50,416 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:50,425 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/361BC40960D746998A9C390C15BAD79C/D62672560F6E4581822146BA2D79E420/.tez/application_1452791810416_0001 doesn't exist and is created
2016-01-14 17:16:50,441 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:50,441 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:50,441 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/361BC40960D746998A9C390C15BAD79C/D62672560F6E4581822146BA2D79E420/.tez/application_1452791810416_0001
2016-01-14 17:16:50,443 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791810416_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:50,454 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:50,454 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:50,455 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:50,455 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:50,455 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/361BC40960D746998A9C390C15BAD79C/D62672560F6E4581822146BA2D79E420/.tez/application_1452791810416_0001/localmode-log-dir
2016-01-14 17:16:50,455 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/361BC40960D746998A9C390C15BAD79C/D62672560F6E4581822146BA2D79E420/.tez/application_1452791810416_0001/localmode-log-dir/history.txt.appattempt_1452791810416_0001_000000, maxErrors=10
2016-01-14 17:16:50,455 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:50,456 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791810416_0001_000000, appSubmitTime=1452791810443, launchTime=1452791810443
2016-01-14 17:16:50,459 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:50,459 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:50,460 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:50,460 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:50289
2016-01-14 17:16:50,461 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:50,462 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:50,463 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791810416_0001_000000, startTime=1452791810463
2016-01-14 17:16:50,463 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:50,463 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:50,541 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:50,541 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:50,542 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...yplatform/ALL_VALUES_SWAP, applicationId=application_1452791810416_0001, dagName=[361BC40960D746998A9C390C15BAD79C/D62672560F6E4581822146BA2D79E420] every/(1/1) ...yplatform/ALL_VALUES_SWAP
2016-01-14 17:16:50,563 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [361BC40960D746998A9C390C15BAD79C/D62672560F6E4581822146BA2D79E420] every/(1/1) ...yplatform/ALL_VALUES_SWAP
2016-01-14 17:16:50,564 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791810416_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/361BC40960D746998A9C390C15BAD79C/D62672560F6E4581822146BA2D79E420/.tez/application_1452791810416_0001/localmode-log-dir/dag_1452791810416_0001_1.dot
2016-01-14 17:16:50,564 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/361BC40960D746998A9C390C15BAD79C/D62672560F6E4581822146BA2D79E420/.tez/application_1452791810416_0001/localmode-log-dir/dag_1452791810416_0001_1-tez-dag.pb.txt
2016-01-14 17:16:50,570 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791810416_0001_1
2016-01-14 17:16:50,570 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:50,570 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [361BC40960D746998A9C390C15BAD79C/D62672560F6E4581822146BA2D79E420] every/(1/1) ...yplatform/ALL_VALUES_SWAP
2016-01-14 17:16:50 Running Dag: dag_1452791810416_0001_1
2016-01-14 17:16:50,581 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791810416_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791810416_0001_1, submitTime=1452791810563
2016-01-14 17:16:50,581 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:50,582 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex BF08EE3212214F97B394FE7754F230F0
2016-01-14 17:16:50,582 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:50,582 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791810416_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791810416_0001_1, initTime=1452791810581
2016-01-14 17:16:50,582 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:50,582 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791810416_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:50,583 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791810416_0001_1][Event:DAG_STARTED]: dagID=dag_1452791810416_0001_1, startTime=1452791810582
2016-01-14 17:16:50,583 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:50,583 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...yplatform/ALL_VALUES_SWAP, applicationId=application_1452791810416_0001, dagName=[361BC40960D746998A9C390C15BAD79C/D62672560F6E4581822146BA2D79E420] every/(1/1) ...yplatform/ALL_VALUES_SWAP
2016-01-14 17:16:50,583 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:50,583 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791810416_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:50,583 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: EAD9612140194164ABBC7DB4D9D801B6 : {965EBD6C468D49369A024970705C484D={InputName=965EBD6C468D49369A024970705C484D}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:50,583 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 965EBD6C468D49369A024970705C484D, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:50,583 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791810416_0001_1_00 [EAD9612140194164ABBC7DB4D9D801B6]
2016-01-14 17:16:50,583 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791810416_0001_1_00 [EAD9612140194164ABBC7DB4D9D801B6]
2016-01-14 17:16:50,584 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791810416_0001_1_00 [EAD9612140194164ABBC7DB4D9D801B6]
2016-01-14 17:16:50,584 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791810416_0001_1_00 [EAD9612140194164ABBC7DB4D9D801B6]. Starting root input initializers: 1
2016-01-14 17:16:50,584 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791810416_0001_1_00 [EAD9612140194164ABBC7DB4D9D801B6] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:50,584 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791810416_0001_1_01 [BF08EE3212214F97B394FE7754F230F0]
2016-01-14 17:16:50,585 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 965EBD6C468D49369A024970705C484D on vertex vertex_1452791810416_0001_1_00 [EAD9612140194164ABBC7DB4D9D801B6]
2016-01-14 17:16:50,587 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:50,587 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:50,587 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791810416_0001_1_01 [BF08EE3212214F97B394FE7754F230F0]
2016-01-14 17:16:50,587 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 965EBD6C468D49369A024970705C484D asking for 1 tasks. Headroom: 732 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:50,587 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791810416_0001_1_01 [BF08EE3212214F97B394FE7754F230F0]
2016-01-14 17:16:50,588 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791810416_0001_1_01 [BF08EE3212214F97B394FE7754F230F0]
2016-01-14 17:16:50,588 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=955DDEFF41AD44EBBFEEA8BBB8E68B27, vertexId=vertex_1452791810416_0001_1_01 [BF08EE3212214F97B394FE7754F230F0], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:50,588 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=955DDEFF41AD44EBBFEEA8BBB8E68B27, vertexId=vertex_1452791810416_0001_1_01 [BF08EE3212214F97B394FE7754F230F0]
2016-01-14 17:16:50,590 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=955DDEFF41AD44EBBFEEA8BBB8E68B27, vertexName=BF08EE3212214F97B394FE7754F230F0, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:50,590 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=955DDEFF41AD44EBBFEEA8BBB8E68B27, vertexId=vertex_1452791810416_0001_1_01 [BF08EE3212214F97B394FE7754F230F0]
2016-01-14 17:16:50,591 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791810416_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BF08EE3212214F97B394FE7754F230F0, vertexId=vertex_1452791810416_0001_1_01, initRequestedTime=1452791810584, initedTime=1452791810591, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:50,591 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:50,591 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791810416_0001_1_01 [BF08EE3212214F97B394FE7754F230F0] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:50,591 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:50,592 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:50,593 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:50,593 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 965EBD6C468D49369A024970705C484D on vertex vertex_1452791810416_0001_1_00 [EAD9612140194164ABBC7DB4D9D801B6]
2016-01-14 17:16:50,593 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791810416_0001_1_00 [EAD9612140194164ABBC7DB4D9D801B6] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:50,593 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {965EBD6C468D49369A024970705C484D=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:50,594 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791810416_0001_1_00 [EAD9612140194164ABBC7DB4D9D801B6] parallelism set to 1
2016-01-14 17:16:50,594 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791810416_0001_1_00 [EAD9612140194164ABBC7DB4D9D801B6] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:50,594 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791810416_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791810416_0001_1_00, eventCount=1
2016-01-14 17:16:50,594 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791810416_0001_1][Event:VERTEX_INITIALIZED]: vertexName=EAD9612140194164ABBC7DB4D9D801B6, vertexId=vertex_1452791810416_0001_1_00, initRequestedTime=1452791810583, initedTime=1452791810594, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:50,594 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:50,595 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791810416_0001_1_00 [EAD9612140194164ABBC7DB4D9D801B6] with distanceFromRoot: 0
2016-01-14 17:16:50,595 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791810416_0001_1_00 [EAD9612140194164ABBC7DB4D9D801B6] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:50,595 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in EAD9612140194164ABBC7DB4D9D801B6
2016-01-14 17:16:50,595 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791810416_0001_1_00 [EAD9612140194164ABBC7DB4D9D801B6]
2016-01-14 17:16:50,595 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:50,596 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791810416_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791810416_0001_1_00, startRequestedTime=1452791810595, startedTime=1452791810595
2016-01-14 17:16:50,596 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791810416_0001_1_00 [EAD9612140194164ABBC7DB4D9D801B6] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:50,596 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:50,596 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791810416_0001_1][Event:TASK_STARTED]: vertexName=EAD9612140194164ABBC7DB4D9D801B6, taskId=task_1452791810416_0001_1_00_000000, scheduledTime=1452791810596, launchTime=1452791810596
2016-01-14 17:16:50,597 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791810416_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:50,597 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791810416_0001_1_00 for vertex: vertex_1452791810416_0001_1_01 [BF08EE3212214F97B394FE7754F230F0] numStartedSources: 1 numSources: 1
2016-01-14 17:16:50,597 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791810416_0001_1_01 [BF08EE3212214F97B394FE7754F230F0] with distanceFromRoot: 1
2016-01-14 17:16:50,597 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791810416_0001_1_00 [EAD9612140194164ABBC7DB4D9D801B6]
2016-01-14 17:16:50,597 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: EAD9612140194164ABBC7DB4D9D801B6 in vertex: BF08EE3212214F97B394FE7754F230F0
2016-01-14 17:16:50,598 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: BF08EE3212214F97B394FE7754F230F0 with 1 source tasks and 1 pending tasks
2016-01-14 17:16:50,598 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:50,598 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791810416_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791810416_0001_1_01, startRequestedTime=1452791810597, startedTime=1452791810597
2016-01-14 17:16:50,598 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791810416_0001_1_01 [BF08EE3212214F97B394FE7754F230F0] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:50,598 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [361BC40960D746998A9C390C15BAD79C/D62672560F6E4581822146BA2D79E420] every/(1/1) ...yplatform/ALL_VALUES_SWAP, VertexName: EAD9612140194164ABBC7DB4D9D801B6, VertexParallelism: 1, TaskAttemptID:attempt_1452791810416_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=965EBD6C468D49369A024970705C484D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=BF08EE3212214F97B394FE7754F230F0, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:50,599 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791810416_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:50,599 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:50,600 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:50,600 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:50,600 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791810416_0001, with dag id: dag_1452791810416_0001_1
2016-01-14 17:16:50,600 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:50,601 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791810416_0001_00_000001
2016-01-14 17:16:50,601 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791810416_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:50,602 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791810416_0001_00_000001
2016-01-14 17:16:50,602 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:50,603 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:50,603 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791810416_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791810416_0001_00_000001, launchTime=1452791810603
2016-01-14 17:16:50,604 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791810416_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:50,605 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791810416_0001_00_000001
2016-01-14 17:16:50,605 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791810416_0001_1_00_000000_0] to container: [container_1452791810416_0001_00_000001]
2016-01-14 17:16:50,605 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791810416_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:50,605 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791810416_0001_00_000001 given task: attempt_1452791810416_0001_1_00_000000_0
2016-01-14 17:16:50,605 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791810416_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791810416_0001_1_00_000000_0
2016-01-14 17:16:50,606 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791810416_0001_1_00_000000_0] started. Is using containerId: [container_1452791810416_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:50,606 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:50,606 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791810416_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=EAD9612140194164ABBC7DB4D9D801B6, taskAttemptId=attempt_1452791810416_0001_1_00_000000_0, startTime=1452791810606, containerId=container_1452791810416_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791810416_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:50,606 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791810416_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:50 Starting to run new task attempt: attempt_1452791810416_0001_1_00_000000_0
2016-01-14 17:16:50,607 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791810416_0001_1_00_000000_0
2016-01-14 17:16:50,607 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:50,607 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791810416_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:50,607 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:50,607 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:50,608 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:50,608 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:50,608 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:50,608 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [361BC40960D746998A9C390C15BAD79C/D62672560F6E4581822146BA2D79E420] every/(1/1) ...yplatform/ALL_VALUES_SWAP, VertexName: EAD9612140194164ABBC7DB4D9D801B6, VertexParallelism: 1, TaskAttemptID:attempt_1452791810416_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=965EBD6C468D49369A024970705C484D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=BF08EE3212214F97B394FE7754F230F0, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:50,608 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=767557632
2016-01-14 17:16:50,611 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791810416_0001_1_00_000000_0
2016-01-14 17:16:50,611 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:50,611 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:50,611 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791810416_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:50,612 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=965EBD6C468D49369A024970705C484D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:50,612 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:50,612 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=BF08EE3212214F97B394FE7754F230F0, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:50,612 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:50,613 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 965EBD6C468D49369A024970705C484D
2016-01-14 17:16:50,613 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 965EBD6C468D49369A024970705C484D
2016-01-14 17:16:50,613 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:50,613 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: BF08EE3212214F97B394FE7754F230F0
2016-01-14 17:16:50,614 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:50,614 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:50,615 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 965EBD6C468D49369A024970705C484D
2016-01-14 17:16:50,615 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 965EBD6C468D49369A024970705C484D
2016-01-14 17:16:50,614 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: BF08EE3212214F97B394FE7754F230F0
2016-01-14 17:16:50,615 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: BF08EE3212214F97B394FE7754F230F0
2016-01-14 17:16:50,617 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:50,620 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: EAD9612140194164ABBC7DB4D9D801B6, ordinal: 0
2016-01-14 17:16:50,620 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EAD9612140194164ABBC7DB4D9D801B6, mem on start (mb), free: 628, total: 732, max: 732
2016-01-14 17:16:50,620 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:50,620 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:50,620 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:50,620 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:50,620 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:50,620 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:50,621 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:50,621 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:50,621 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:50,621 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 767557632, TotalAvailable: 514263613, TotalRequested/TotalJVMHeap:0.14
2016-01-14 17:16:50,621 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 965EBD6C468D49369A024970705C484D, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:50,621 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, BF08EE3212214F97B394FE7754F230F0, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:50,622 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:50,622 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 965EBD6C468D49369A024970705C484D being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:50,622 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:50,622 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:50,622 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 965EBD6C468D49369A024970705C484D
2016-01-14 17:16:50,622 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:50,622 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 965EBD6C468D49369A024970705C484D
2016-01-14 17:16:50,623 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:50,623 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791810416_0001_1_00_000000_0
2016-01-14 17:16:50,623 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:50,623 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@734398b9
2016-01-14 17:16:50,624 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@734398b9
2016-01-14 17:16:50,624 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:50,624 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:50,626 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:50,626 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 965EBD6C468D49369A024970705C484D
2016-01-14 17:16:50,627 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 95107D1412394E03BEA96CAB9DD7296A
2016-01-14 17:16:50,627 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 95107D1412394E03BEA96CAB9DD7296A
2016-01-14 17:16:50,627 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@7fa256ea; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@7af85e1c; comparator=cascading.tuple.hadoop.util.TupleComparator@47eabf56; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:50,628 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:50,628 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 17:16:50,628 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 17:16:50,628 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:50,628 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:50,645 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:50,645 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:50,646 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:50,646 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:50,646 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:50,647 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:50,647 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:50,647 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: EAD9612140194164ABBC7DB4D9D801B6, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:50,650 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791810416_0001_1_00_000000_0
2016-01-14 17:16:50,650 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EAD9612140194164ABBC7DB4D9D801B6, mem on close (mb), free: 526, total: 732, max: 732
2016-01-14 17:16:50,650 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:50,650 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:50,650 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:50,650 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:50,653 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:50 Completed running task attempt: attempt_1452791810416_0001_1_00_000000_0
2016-01-14 17:16:50,656 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791810416_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:50,656 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=767557632, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791810647, Process_Duration=2, Process_End_Time=1452791810649, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:50,656 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791810416_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791810416_0001_1_00, eventCount=1
2016-01-14 17:16:50,657 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:50,657 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:50,657 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791810416_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=EAD9612140194164ABBC7DB4D9D801B6, taskAttemptId=attempt_1452791810416_0001_1_00_000000_0, startTime=1452791810606, finishTime=1452791810657, timeTaken=51, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=767557632, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791810647, Process_Duration=2, Process_End_Time=1452791810649, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:50,657 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791810416_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:50,657 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791810416_0001_1_00_000000_0
2016-01-14 17:16:50,657 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791810416_0001_00_000001
2016-01-14 17:16:50,657 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791810416_0001_1][Event:TASK_FINISHED]: vertexName=EAD9612140194164ABBC7DB4D9D801B6, taskId=task_1452791810416_0001_1_00_000000, startTime=1452791810606, finishTime=1452791810657, timeTaken=51, status=SUCCEEDED, successfulAttemptID=attempt_1452791810416_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=767557632, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791810647, Process_Duration=2, Process_End_Time=1452791810649, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:50,657 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:50,657 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:50,658 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:50,657 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791810416_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:50,658 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791810416_0001_1_00 [EAD9612140194164ABBC7DB4D9D801B6] : 1
2016-01-14 17:16:50,658 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791810416_0001_1_00 [EAD9612140194164ABBC7DB4D9D801B6]
2016-01-14 17:16:50,659 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:50,659 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791810416_0001_1][Event:VERTEX_FINISHED]: vertexName=EAD9612140194164ABBC7DB4D9D801B6, vertexId=vertex_1452791810416_0001_1_00, initRequestedTime=1452791810583, initedTime=1452791810594, startRequestedTime=1452791810595, startedTime=1452791810595, finishTime=1452791810658, timeTaken=63, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=767557632, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791810647, Process_Duration=2, Process_End_Time=1452791810649, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791810606, firstTasksToStart=[ task_1452791810416_0001_1_00_000000 ], lastTaskFinishTime=1452791810657, lastTasksToFinish=[ task_1452791810416_0001_1_00_000000 ], minTaskDuration=51, maxTaskDuration=51, avgTaskDuration=51.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791810416_0001_1_00_000000 ], longestDurationTasks=[ task_1452791810416_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:50,659 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791810416_0001_1_00 [EAD9612140194164ABBC7DB4D9D801B6] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:50,659 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791810416_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:50,659 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791810416_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791810416_0001_00_000001, stoppedTime=1452791810659, exitStatus=0
2016-01-14 17:16:50,659 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:50,659 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791810416_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:50,659 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791810416_0001_1_01 [BF08EE3212214F97B394FE7754F230F0] attempt: attempt_1452791810416_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:50,659 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: BF08EE3212214F97B394FE7754F230F0
2016-01-14 17:16:50,660 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791810416_0001_1_01 [BF08EE3212214F97B394FE7754F230F0]
2016-01-14 17:16:50,660 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791810416_0001_1_00 [EAD9612140194164ABBC7DB4D9D801B6] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:50,660 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791810416_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:50,660 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791810416_0001_1][Event:TASK_STARTED]: vertexName=BF08EE3212214F97B394FE7754F230F0, taskId=task_1452791810416_0001_1_01_000000, scheduledTime=1452791810660, launchTime=1452791810660
2016-01-14 17:16:50,660 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791810416_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:50,660 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791810416_0001_1_01 [BF08EE3212214F97B394FE7754F230F0]
2016-01-14 17:16:50,660 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791810416_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:50,660 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [361BC40960D746998A9C390C15BAD79C/D62672560F6E4581822146BA2D79E420] every/(1/1) ...yplatform/ALL_VALUES_SWAP, VertexName: BF08EE3212214F97B394FE7754F230F0, VertexParallelism: 1, TaskAttemptID:attempt_1452791810416_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EAD9612140194164ABBC7DB4D9D801B6, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=955DDEFF41AD44EBBFEEA8BBB8E68B27, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:50,661 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791810416_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:50,661 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:50,661 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:50,661 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791810416_0001_00_000002
2016-01-14 17:16:50,661 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791810416_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:50,662 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791810416_0001_00_000002
2016-01-14 17:16:50,663 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:50,663 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791810416_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:50,663 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:50,664 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791810416_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791810416_0001_00_000002, launchTime=1452791810663
2016-01-14 17:16:50,664 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791810416_0001_00_000002
2016-01-14 17:16:50,664 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791810416_0001_1_01_000000_0] to container: [container_1452791810416_0001_00_000002]
2016-01-14 17:16:50,665 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791810416_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:50,665 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791810416_0001_1_01_000000_0] started. Is using containerId: [container_1452791810416_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:50,665 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791810416_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BF08EE3212214F97B394FE7754F230F0, taskAttemptId=attempt_1452791810416_0001_1_01_000000_0, startTime=1452791810665, containerId=container_1452791810416_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791810416_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:50,665 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:50,665 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791810416_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:50,665 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791810416_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:50,666 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791810416_0001_00_000002 given task: attempt_1452791810416_0001_1_01_000000_0
2016-01-14 17:16:50,666 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791810416_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791810416_0001_1_01_000000_0
2016-01-14 17:16:50 Starting to run new task attempt: attempt_1452791810416_0001_1_01_000000_0
2016-01-14 17:16:50,667 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791810416_0001_1_01_000000_0
2016-01-14 17:16:50,667 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:50,667 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:50,667 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:50,668 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:50,668 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:50,668 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:50,668 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [361BC40960D746998A9C390C15BAD79C/D62672560F6E4581822146BA2D79E420] every/(1/1) ...yplatform/ALL_VALUES_SWAP, VertexName: BF08EE3212214F97B394FE7754F230F0, VertexParallelism: 1, TaskAttemptID:attempt_1452791810416_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EAD9612140194164ABBC7DB4D9D801B6, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=955DDEFF41AD44EBBFEEA8BBB8E68B27, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:50,669 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=767557632
2016-01-14 17:16:50,669 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791810416_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:50,669 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791810416_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:50,669 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791810416_0001_00_000001, Exiting
2016-01-14 17:16:50,669 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791810416_0001_00_000001
2016-01-14 17:16:50,669 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791810416_0001_00_000001
2016-01-14 17:16:50,670 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791810416_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:50,670 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791810416_0001_00_000001 completed successfully
2016-01-14 17:16:50,670 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791810416_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:50,670 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791810416_0001_1_01_000000_0
2016-01-14 17:16:50,671 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:50,671 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:50,671 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791810416_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:50,671 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=EAD9612140194164ABBC7DB4D9D801B6, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:50,671 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:50,672 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=955DDEFF41AD44EBBFEEA8BBB8E68B27, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:50,674 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:50,674 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:50,674 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 955DDEFF41AD44EBBFEEA8BBB8E68B27
2016-01-14 17:16:50,675 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:50,675 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 955DDEFF41AD44EBBFEEA8BBB8E68B27
2016-01-14 17:16:50,677 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:50,678 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: EAD9612140194164ABBC7DB4D9D801B6
2016-01-14 17:16:50,678 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BF08EE3212214F97B394FE7754F230F0, ordinal: 1
2016-01-14 17:16:50,678 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BF08EE3212214F97B394FE7754F230F0, mem on start (mb), free: 521, total: 732, max: 732
2016-01-14 17:16:50,678 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:50,679 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:50,680 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 690801856, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:50,680 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:50,680 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 690801856, type: INPUT, componentVertexName: EAD9612140194164ABBC7DB4D9D801B6
2016-01-14 17:16:50,680 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: EAD9612140194164ABBC7DB4D9D801B6
2016-01-14 17:16:50,681 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:50,685 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:50,685 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 955DDEFF41AD44EBBFEEA8BBB8E68B27
2016-01-14 17:16:50,686 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:50,686 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:50,686 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:50,686 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:50,686 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:50,686 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:50,686 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 690801856, TotalRequestedScaled: 6.376632516923077E8, TotalJVMHeap: 767557632, TotalAvailable: 514263613, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:50,686 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 955DDEFF41AD44EBBFEEA8BBB8E68B27, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:50,686 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, EAD9612140194164ABBC7DB4D9D801B6, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=690801856, allocated=514263613
2016-01-14 17:16:50,686 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:50,687 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: EAD9612140194164ABBC7DB4D9D801B6 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:50,687 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:50,687 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:50,687 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: EAD9612140194164ABBC7DB4D9D801B6
2016-01-14 17:16:50,687 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:50,687 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:50,688 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: EAD9612140194164ABBC7DB4D9D801B6 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:50,688 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=690801856, postMergeMem=0, RuntimeTotalAvailable=514263613. Updated to: ShuffleMem=514263613, postMergeMem: 0
2016-01-14 17:16:50,688 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=514263613, maxSingleShuffleLimit=128565904, mergeThreshold=462837248, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:50,688 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:50,688 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: EAD9612140194164ABBC7DB4D9D801B6
2016-01-14 17:16:50,689 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:50,689 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791810416_0001_1_01_000000_0
2016-01-14 17:16:50,692 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:50,692 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 95107D1412394E03BEA96CAB9DD7296A
2016-01-14 17:16:50,692 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_SWAP"], id: 955DDEFF41AD44EBBFEEA8BBB8E68B27
2016-01-14 17:16:50,693 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_SWAP"]
2016-01-14 17:16:50,693 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791810416_0001_1_00_000000_0_10315, runDuration: 42023, ]
2016-01-14 17:16:50,693 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 95107D1412394E03BEA96CAB9DD7296A, for 1 inputs
2016-01-14 17:16:50,694 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791810416_0001_1_00_000000_0_10315] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791810694, TimeTaken=0, Rate=0.00 MB/s
2016-01-14 17:16:50,694 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : EAD9612140194164ABBC7DB4D9D801B6
2016-01-14 17:16:50,694 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:50,694 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [EAD9612140194164ABBC7DB4D9D801B6] #53 in 1ms
2016-01-14 17:16:50,695 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [EAD9612140194164ABBC7DB4D9D801B6] #53, status:false, isInterrupted:false
2016-01-14 17:16:50,695 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:50,695 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:50,696 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:50,696 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:50,696 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:50,696 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : EAD9612140194164ABBC7DB4D9D801B6
2016-01-14 17:16:50,696 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:50,696 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BF08EE3212214F97B394FE7754F230F0, all 1 inputs ready in: 00:00:00.003
2016-01-14 17:16:50,697 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@3019169d; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@3f1bb616
2016-01-14 17:16:50,697 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:50,698 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(92)) - caught throwable
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:50,698 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:50,698 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:50,699 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791810416_0001_1_01_000000_0
2016-01-14 17:16:50,699 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791810416_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:50,700 INFO  task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1452791810416_0001_1_01_000000_0
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:50,701 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=767557632, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791810696, Process_Duration=4, Process_End_Time=1452791810700, Read_Duration=0, Tuples_Read=1]]
2016-01-14 17:16:50,701 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:50,701 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791810416_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BF08EE3212214F97B394FE7754F230F0, taskAttemptId=attempt_1452791810416_0001_1_01_000000_0, startTime=1452791810665, finishTime=1452791810701, timeTaken=36, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=767557632, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791810696, Process_Duration=4, Process_End_Time=1452791810700, Read_Duration=0, Tuples_Read=1
2016-01-14 17:16:50,702 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791810416_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-01-14 17:16:50,701 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BF08EE3212214F97B394FE7754F230F0, mem on close (mb), free: 515, total: 732, max: 732
2016-01-14 17:16:50,702 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=BF08EE3212214F97B394FE7754F230F0, index=1
2016-01-14 17:16:50,702 INFO  impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1452791810416_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-01-14 17:16:50,703 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791810416_0001_1][Event:TASK_FINISHED]: vertexName=BF08EE3212214F97B394FE7754F230F0, taskId=task_1452791810416_0001_1_01_000000, startTime=1452791810665, finishTime=1452791810703, timeTaken=38, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
], counters=Counters: 0
2016-01-14 17:16:50,703 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:50,703 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: EAD9612140194164ABBC7DB4D9D801B6
2016-01-14 17:16:50,703 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791810416_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-01-14 17:16:50,704 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791810416_0001_1_01 [BF08EE3212214F97B394FE7754F230F0] : 1
2016-01-14 17:16:50,704 INFO  impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1452791810416_0001_1_01 [BF08EE3212214F97B394FE7754F230F0] because task failed: task_1452791810416_0001_1_01_000000
2016-01-14 17:16:50,704 INFO  impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1452791810416_0001_1_01 [BF08EE3212214F97B394FE7754F230F0] due to trigger: OWN_TASK_FAILURE
2016-01-14 17:16:50,704 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-01-14 17:16:50,704 INFO  impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1452791810416_0001_1_01 [BF08EE3212214F97B394FE7754F230F0]
2016-01-14 17:16:50,704 INFO  impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=955DDEFF41AD44EBBFEEA8BBB8E68B27, vertexId=vertex_1452791810416_0001_1_01 [BF08EE3212214F97B394FE7754F230F0]
2016-01-14 17:16:50,704 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:50,704 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:50,705 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:50,705 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:50,705 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:50,706 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791810416_0001_1][Event:VERTEX_FINISHED]: vertexName=BF08EE3212214F97B394FE7754F230F0, vertexId=vertex_1452791810416_0001_1_01, initRequestedTime=1452791810584, initedTime=1452791810591, startRequestedTime=1452791810597, startedTime=1452791810597, finishTime=1452791810705, timeTaken=108, status=FAILED, diagnostics=Task failed, taskId=task_1452791810416_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1452791810416_0001_1_01 [BF08EE3212214F97B394FE7754F230F0] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1452791810665, firstTasksToStart=[ task_1452791810416_0001_1_01_000000 ], lastTaskFinishTime=1452791810701, lastTasksToFinish=[ task_1452791810416_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[  ], longestDurationTasks=[  ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-01-14 17:16:50,706 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791810416_0001_1_01 [BF08EE3212214F97B394FE7754F230F0] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-01-14 17:16:50,706 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791810416_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791810416_0001_00_000002, stoppedTime=1452791810706, exitStatus=0
2016-01-14 17:16:50,706 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791810416_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:50,706 INFO  node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1452791810416_0001_1_01_000000_0 failed: true container: container_1452791810416_0001_00_000002 numFailedTAs: 0
2016-01-14 17:16:50,706 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791810416_0001_1_01 [BF08EE3212214F97B394FE7754F230F0] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-01-14 17:16:50,706 INFO  impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-01-14 17:16:50,706 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791810416_0001_1
2016-01-14 17:16:50,707 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 955DDEFF41AD44EBBFEEA8BBB8E68B27 for vertex: vertex_1452791810416_0001_1_01 [BF08EE3212214F97B394FE7754F230F0]
2016-01-14 17:16:50,706 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:50,707 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=BF08EE3212214F97B394FE7754F230F0, sourceVertex=EAD9612140194164ABBC7DB4D9D801B6
2016-01-14 17:16:50,706 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:50,707 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=BF08EE3212214F97B394FE7754F230F0, sourceVertex=955DDEFF41AD44EBBFEEA8BBB8E68B27
2016-01-14 17:16:50,707 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1452791810416_0001_1_00 [EAD9612140194164ABBC7DB4D9D801B6]
2016-01-14 17:16:50,707 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:50,709 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791810416_0001_1, queueSize=0
2016-01-14 17:16:50,709 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:50,709 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791810416_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791810416_0001_1, startTime=1452791810582, finishTime=1452791810708, timeTaken=126, status=FAILED, diagnostics=Vertex failed, vertexName=BF08EE3212214F97B394FE7754F230F0, vertexId=vertex_1452791810416_0001_1_01, diagnostics=[Task failed, taskId=task_1452791810416_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1452791810416_0001_1_01 [BF08EE3212214F97B394FE7754F230F0] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=767557632, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791810647, Process_Duration=2, Process_End_Time=1452791810649, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:50,709 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791810416_0001_1 finished with state: FAILED
2016-01-14 17:16:50,709 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791810416_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:50,710 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791810416_0001_00_000002. Relying on regular task shutdown for it to end
2016-01-14 17:16:50 Completed Dag: dag_1452791810416_0001_1
2016-01-14 17:16:50,710 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791810416_0001_1, dagState=FAILED
2016-01-14 17:16:50,710 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791810416_0001_1_post
2016-01-14 17:16:50,710 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:50,710 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:50,710 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791810416_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:50,711 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791810416_0001_00_000002
2016-01-14 17:16:50,711 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791810416_0001_00_000002
2016-01-14 17:16:50,711 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791810416_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-01-14 17:16:50,711 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1452791810416_0001_00_000002 completed but with errors
2016-01-14 17:16:50,712 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1452791810416_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-01-14 17:16:50,712 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791810416_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:50,712 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791810416_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-01-14 17:16:50,719 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...yplatform/ALL_VALUES_SWAP, applicationId=application_1452791810416_0001
2016-01-14 17:16:50,721 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:50,721 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:50,721 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:50,721 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:50,721 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:50,721 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:50,722 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:50,722 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:50,722 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:50,722 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:50,722 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:50,723 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:50,723 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:50,728 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/361BC40960D746998A9C390C15BAD79C/D62672560F6E4581822146BA2D79E420/.tez/application_1452791810416_0001
2016-01-14 17:16:50,728 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping all jobs
2016-01-14 17:16:50,728 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping: (1/1) ...yplatform/ALL_VALUES_SWAP
2016-01-14 17:16:50,729 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopped all jobs
2016-01-14 17:16:50,729 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.330
2016-01-14 17:16:50,729 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_VALUES_SWAP
2016-01-14 17:16:50,759 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:50,760 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:50,797 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.008
2016-01-14 17:16:50,801 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-01-14 17:16:50,802 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:50,802 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:50,802 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:50,815 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:50,819 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_ARGS_-1
2016-01-14 17:16:50,876 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:50,877 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:50,877 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_ARGS_-1"]
2016-01-14 17:16:50,877 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:50,877 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:50,877 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:50,878 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...emblyplatform/ALL_ARGS_-1
2016-01-14 17:16:50,878 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:50,889 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:50,894 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:50,894 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:50,894 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:50,903 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/14DF9C41F50948FB955D6B0EB3F9510E/CFAFB7C034A54E8397FCC05E2AE3B1C4/.tez/application_1452791810894_0001 doesn't exist and is created
2016-01-14 17:16:50,919 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:50,919 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:50,920 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/14DF9C41F50948FB955D6B0EB3F9510E/CFAFB7C034A54E8397FCC05E2AE3B1C4/.tez/application_1452791810894_0001
2016-01-14 17:16:50,922 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791810894_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:50,931 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:50,932 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:50,933 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:50,933 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:50,933 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/14DF9C41F50948FB955D6B0EB3F9510E/CFAFB7C034A54E8397FCC05E2AE3B1C4/.tez/application_1452791810894_0001/localmode-log-dir
2016-01-14 17:16:50,933 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/14DF9C41F50948FB955D6B0EB3F9510E/CFAFB7C034A54E8397FCC05E2AE3B1C4/.tez/application_1452791810894_0001/localmode-log-dir/history.txt.appattempt_1452791810894_0001_000000, maxErrors=10
2016-01-14 17:16:50,933 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:50,934 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791810894_0001_000000, appSubmitTime=1452791810922, launchTime=1452791810922
2016-01-14 17:16:50,936 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:50,936 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:50,937 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:50,938 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:44772
2016-01-14 17:16:50,939 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:50,940 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:50,942 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791810894_0001_000000, startTime=1452791810942
2016-01-14 17:16:50,942 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:50,942 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:51,020 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:51,020 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:51,020 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...emblyplatform/ALL_ARGS_-1, applicationId=application_1452791810894_0001, dagName=[14DF9C41F50948FB955D6B0EB3F9510E/CFAFB7C034A54E8397FCC05E2AE3B1C4] every/(1/1) ...emblyplatform/ALL_ARGS_-1
2016-01-14 17:16:51,035 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [14DF9C41F50948FB955D6B0EB3F9510E/CFAFB7C034A54E8397FCC05E2AE3B1C4] every/(1/1) ...emblyplatform/ALL_ARGS_-1
2016-01-14 17:16:51,036 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791810894_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/14DF9C41F50948FB955D6B0EB3F9510E/CFAFB7C034A54E8397FCC05E2AE3B1C4/.tez/application_1452791810894_0001/localmode-log-dir/dag_1452791810894_0001_1.dot
2016-01-14 17:16:51,037 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/14DF9C41F50948FB955D6B0EB3F9510E/CFAFB7C034A54E8397FCC05E2AE3B1C4/.tez/application_1452791810894_0001/localmode-log-dir/dag_1452791810894_0001_1-tez-dag.pb.txt
2016-01-14 17:16:51,045 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791810894_0001_1
2016-01-14 17:16:51,045 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:51,045 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [14DF9C41F50948FB955D6B0EB3F9510E/CFAFB7C034A54E8397FCC05E2AE3B1C4] every/(1/1) ...emblyplatform/ALL_ARGS_-1
2016-01-14 17:16:51 Running Dag: dag_1452791810894_0001_1
2016-01-14 17:16:51,057 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791810894_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791810894_0001_1, submitTime=1452791811035
2016-01-14 17:16:51,057 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:51,058 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex D562CB8D48274A37896EA06F4389CF7F
2016-01-14 17:16:51,059 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:51,059 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791810894_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791810894_0001_1, initTime=1452791811058
2016-01-14 17:16:51,059 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:51,059 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791810894_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:51,060 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791810894_0001_1][Event:DAG_STARTED]: dagID=dag_1452791810894_0001_1, startTime=1452791811060
2016-01-14 17:16:51,060 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:51,060 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:51,060 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791810894_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:51,060 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...emblyplatform/ALL_ARGS_-1, applicationId=application_1452791810894_0001, dagName=[14DF9C41F50948FB955D6B0EB3F9510E/CFAFB7C034A54E8397FCC05E2AE3B1C4] every/(1/1) ...emblyplatform/ALL_ARGS_-1
2016-01-14 17:16:51,060 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: BEF13210C18F42329A5C87C1676047D3 : {92FB76F4412D42E1BD223747AC71C31F={InputName=92FB76F4412D42E1BD223747AC71C31F}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:51,061 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 92FB76F4412D42E1BD223747AC71C31F, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:51,061 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791810894_0001_1_00 [BEF13210C18F42329A5C87C1676047D3]
2016-01-14 17:16:51,061 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791810894_0001_1_00 [BEF13210C18F42329A5C87C1676047D3]
2016-01-14 17:16:51,061 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791810894_0001_1_00 [BEF13210C18F42329A5C87C1676047D3]
2016-01-14 17:16:51,061 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791810894_0001_1_00 [BEF13210C18F42329A5C87C1676047D3]. Starting root input initializers: 1
2016-01-14 17:16:51,062 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791810894_0001_1_00 [BEF13210C18F42329A5C87C1676047D3] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:51,062 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791810894_0001_1_01 [D562CB8D48274A37896EA06F4389CF7F]
2016-01-14 17:16:51,065 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:51,065 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791810894_0001_1_01 [D562CB8D48274A37896EA06F4389CF7F]
2016-01-14 17:16:51,065 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791810894_0001_1_01 [D562CB8D48274A37896EA06F4389CF7F]
2016-01-14 17:16:51,066 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791810894_0001_1_01 [D562CB8D48274A37896EA06F4389CF7F]
2016-01-14 17:16:51,066 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=F7EE78ED6EB443F794B0E45D5E1CED9C, vertexId=vertex_1452791810894_0001_1_01 [D562CB8D48274A37896EA06F4389CF7F], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:51,066 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=F7EE78ED6EB443F794B0E45D5E1CED9C, vertexId=vertex_1452791810894_0001_1_01 [D562CB8D48274A37896EA06F4389CF7F]
2016-01-14 17:16:51,067 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=F7EE78ED6EB443F794B0E45D5E1CED9C, vertexName=D562CB8D48274A37896EA06F4389CF7F, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:51,068 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=F7EE78ED6EB443F794B0E45D5E1CED9C, vertexId=vertex_1452791810894_0001_1_01 [D562CB8D48274A37896EA06F4389CF7F]
2016-01-14 17:16:51,069 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:51,069 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 92FB76F4412D42E1BD223747AC71C31F on vertex vertex_1452791810894_0001_1_00 [BEF13210C18F42329A5C87C1676047D3]
2016-01-14 17:16:51,071 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:51,071 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 92FB76F4412D42E1BD223747AC71C31F asking for 1 tasks. Headroom: 731 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:51,071 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:51,071 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:51,069 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791810894_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D562CB8D48274A37896EA06F4389CF7F, vertexId=vertex_1452791810894_0001_1_01, initRequestedTime=1452791811062, initedTime=1452791811069, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:51,072 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791810894_0001_1_01 [D562CB8D48274A37896EA06F4389CF7F] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:51,073 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:51,073 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 92FB76F4412D42E1BD223747AC71C31F on vertex vertex_1452791810894_0001_1_00 [BEF13210C18F42329A5C87C1676047D3]
2016-01-14 17:16:51,073 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791810894_0001_1_00 [BEF13210C18F42329A5C87C1676047D3] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:51,073 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {92FB76F4412D42E1BD223747AC71C31F=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:51,073 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791810894_0001_1_00 [BEF13210C18F42329A5C87C1676047D3] parallelism set to 1
2016-01-14 17:16:51,073 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791810894_0001_1_00 [BEF13210C18F42329A5C87C1676047D3] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:51,074 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791810894_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791810894_0001_1_00, eventCount=1
2016-01-14 17:16:51,074 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:51,074 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791810894_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BEF13210C18F42329A5C87C1676047D3, vertexId=vertex_1452791810894_0001_1_00, initRequestedTime=1452791811060, initedTime=1452791811074, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:51,074 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791810894_0001_1_00 [BEF13210C18F42329A5C87C1676047D3] with distanceFromRoot: 0
2016-01-14 17:16:51,074 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791810894_0001_1_00 [BEF13210C18F42329A5C87C1676047D3] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:51,075 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in BEF13210C18F42329A5C87C1676047D3
2016-01-14 17:16:51,075 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791810894_0001_1_00 [BEF13210C18F42329A5C87C1676047D3]
2016-01-14 17:16:51,075 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791810894_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791810894_0001_1_00, startRequestedTime=1452791811074, startedTime=1452791811074
2016-01-14 17:16:51,075 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:51,075 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791810894_0001_1_00 [BEF13210C18F42329A5C87C1676047D3] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:51,076 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:51,076 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791810894_0001_1][Event:TASK_STARTED]: vertexName=BEF13210C18F42329A5C87C1676047D3, taskId=task_1452791810894_0001_1_00_000000, scheduledTime=1452791811075, launchTime=1452791811075
2016-01-14 17:16:51,076 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791810894_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:51,076 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791810894_0001_1_00 for vertex: vertex_1452791810894_0001_1_01 [D562CB8D48274A37896EA06F4389CF7F] numStartedSources: 1 numSources: 1
2016-01-14 17:16:51,076 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791810894_0001_1_01 [D562CB8D48274A37896EA06F4389CF7F] with distanceFromRoot: 1
2016-01-14 17:16:51,076 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791810894_0001_1_00 [BEF13210C18F42329A5C87C1676047D3]
2016-01-14 17:16:51,077 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: BEF13210C18F42329A5C87C1676047D3 in vertex: D562CB8D48274A37896EA06F4389CF7F
2016-01-14 17:16:51,077 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: D562CB8D48274A37896EA06F4389CF7F with 1 source tasks and 1 pending tasks
2016-01-14 17:16:51,077 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791810894_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791810894_0001_1_01, startRequestedTime=1452791811077, startedTime=1452791811077
2016-01-14 17:16:51,077 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:51,077 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791810894_0001_1_01 [D562CB8D48274A37896EA06F4389CF7F] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:51,078 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [14DF9C41F50948FB955D6B0EB3F9510E/CFAFB7C034A54E8397FCC05E2AE3B1C4] every/(1/1) ...emblyplatform/ALL_ARGS_-1, VertexName: BEF13210C18F42329A5C87C1676047D3, VertexParallelism: 1, TaskAttemptID:attempt_1452791810894_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=92FB76F4412D42E1BD223747AC71C31F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D562CB8D48274A37896EA06F4389CF7F, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:51,078 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791810894_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:51,078 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:51,078 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791810894_0001, with dag id: dag_1452791810894_0001_1
2016-01-14 17:16:51,079 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:51,079 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:51,079 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:51,080 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791810894_0001_00_000001
2016-01-14 17:16:51,080 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791810894_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:51,081 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791810894_0001_00_000001
2016-01-14 17:16:51,081 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:51,082 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:51,082 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791810894_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:51,083 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791810894_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791810894_0001_00_000001, launchTime=1452791811082
2016-01-14 17:16:51,084 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791810894_0001_00_000001
2016-01-14 17:16:51,084 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791810894_0001_1_00_000000_0] to container: [container_1452791810894_0001_00_000001]
2016-01-14 17:16:51,085 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791810894_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:51,085 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791810894_0001_1_00_000000_0] started. Is using containerId: [container_1452791810894_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:51,085 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791810894_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BEF13210C18F42329A5C87C1676047D3, taskAttemptId=attempt_1452791810894_0001_1_00_000000_0, startTime=1452791811085, containerId=container_1452791810894_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791810894_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:51,085 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791810894_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:51,085 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:51,086 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791810894_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:51,086 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791810894_0001_00_000001 given task: attempt_1452791810894_0001_1_00_000000_0
2016-01-14 17:16:51,086 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791810894_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791810894_0001_1_00_000000_0
2016-01-14 17:16:51 Starting to run new task attempt: attempt_1452791810894_0001_1_00_000000_0
2016-01-14 17:16:51,087 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791810894_0001_1_00_000000_0
2016-01-14 17:16:51,087 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:51,087 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:51,087 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:51,087 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:51,088 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:51,088 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:51,088 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [14DF9C41F50948FB955D6B0EB3F9510E/CFAFB7C034A54E8397FCC05E2AE3B1C4] every/(1/1) ...emblyplatform/ALL_ARGS_-1, VertexName: BEF13210C18F42329A5C87C1676047D3, VertexParallelism: 1, TaskAttemptID:attempt_1452791810894_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=92FB76F4412D42E1BD223747AC71C31F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D562CB8D48274A37896EA06F4389CF7F, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:51,088 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=766509056
2016-01-14 17:16:51,089 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791810894_0001_1_00_000000_0
2016-01-14 17:16:51,089 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:51,089 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:51,089 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791810894_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:51,090 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:51,089 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=92FB76F4412D42E1BD223747AC71C31F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:51,090 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:51,090 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 92FB76F4412D42E1BD223747AC71C31F
2016-01-14 17:16:51,090 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 92FB76F4412D42E1BD223747AC71C31F
2016-01-14 17:16:51,092 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:51,092 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 92FB76F4412D42E1BD223747AC71C31F
2016-01-14 17:16:51,092 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 92FB76F4412D42E1BD223747AC71C31F
2016-01-14 17:16:51,092 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=D562CB8D48274A37896EA06F4389CF7F, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:51,092 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:51,094 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:51,094 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: D562CB8D48274A37896EA06F4389CF7F
2016-01-14 17:16:51,097 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:51,097 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: D562CB8D48274A37896EA06F4389CF7F
2016-01-14 17:16:51,097 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BEF13210C18F42329A5C87C1676047D3, ordinal: 0
2016-01-14 17:16:51,097 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BEF13210C18F42329A5C87C1676047D3, mem on start (mb), free: 628, total: 731, max: 731
2016-01-14 17:16:51,097 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:51,098 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:51,098 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:51,097 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: D562CB8D48274A37896EA06F4389CF7F
2016-01-14 17:16:51,098 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:51,098 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:51,098 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:51,098 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:51,099 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:51,099 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:51,099 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 766509056, TotalAvailable: 513561067, TotalRequested/TotalJVMHeap:0.14
2016-01-14 17:16:51,099 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 92FB76F4412D42E1BD223747AC71C31F, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:51,099 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, D562CB8D48274A37896EA06F4389CF7F, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:51,099 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:51,100 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 92FB76F4412D42E1BD223747AC71C31F being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:51,100 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:51,100 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:51,100 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 92FB76F4412D42E1BD223747AC71C31F
2016-01-14 17:16:51,100 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:51,101 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 92FB76F4412D42E1BD223747AC71C31F
2016-01-14 17:16:51,101 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:51,101 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791810894_0001_1_00_000000_0
2016-01-14 17:16:51,103 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:51,103 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:51,103 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 92FB76F4412D42E1BD223747AC71C31F
2016-01-14 17:16:51,104 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 073D4B0EE49245D1897EBF487B243A77
2016-01-14 17:16:51,104 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 073D4B0EE49245D1897EBF487B243A77
2016-01-14 17:16:51,104 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@39090ad9
2016-01-14 17:16:51,104 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@a749d0; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@66def705; comparator=cascading.tuple.hadoop.util.TupleComparator@382b9e06; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:51,105 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:51,105 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:51,105 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:51,105 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@39090ad9
2016-01-14 17:16:51,105 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:51,105 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:51,119 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:51,119 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:51,119 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:51,119 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:51,119 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:51,121 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:51,121 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:51,121 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BEF13210C18F42329A5C87C1676047D3, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:51,124 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791810894_0001_1_00_000000_0
2016-01-14 17:16:51,124 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BEF13210C18F42329A5C87C1676047D3, mem on close (mb), free: 526, total: 731, max: 731
2016-01-14 17:16:51,125 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:51,125 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:51,125 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:51,125 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:51,127 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:51 Completed running task attempt: attempt_1452791810894_0001_1_00_000000_0
2016-01-14 17:16:51,129 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791810894_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:51,130 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791810894_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791810894_0001_1_00, eventCount=1
2016-01-14 17:16:51,130 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791810894_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BEF13210C18F42329A5C87C1676047D3, taskAttemptId=attempt_1452791810894_0001_1_00_000000_0, startTime=1452791811085, finishTime=1452791811130, timeTaken=45, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=766509056, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791811122, Process_Duration=2, Process_End_Time=1452791811124, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:51,130 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791810894_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:51,131 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791810894_0001_1_00_000000_0
2016-01-14 17:16:51,131 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:51,131 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:51,131 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791810894_0001_1][Event:TASK_FINISHED]: vertexName=BEF13210C18F42329A5C87C1676047D3, taskId=task_1452791810894_0001_1_00_000000, startTime=1452791811085, finishTime=1452791811131, timeTaken=46, status=SUCCEEDED, successfulAttemptID=attempt_1452791810894_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=766509056, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791811122, Process_Duration=2, Process_End_Time=1452791811124, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:51,131 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791810894_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:51,131 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:51,131 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791810894_0001_1_00 [BEF13210C18F42329A5C87C1676047D3] : 1
2016-01-14 17:16:51,131 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791810894_0001_1_00 [BEF13210C18F42329A5C87C1676047D3]
2016-01-14 17:16:51,132 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:51,132 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791810894_0001_1][Event:VERTEX_FINISHED]: vertexName=BEF13210C18F42329A5C87C1676047D3, vertexId=vertex_1452791810894_0001_1_00, initRequestedTime=1452791811060, initedTime=1452791811074, startRequestedTime=1452791811074, startedTime=1452791811074, finishTime=1452791811131, timeTaken=57, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=766509056, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791811122, Process_Duration=2, Process_End_Time=1452791811124, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791811085, firstTasksToStart=[ task_1452791810894_0001_1_00_000000 ], lastTaskFinishTime=1452791811130, lastTasksToFinish=[ task_1452791810894_0001_1_00_000000 ], minTaskDuration=45, maxTaskDuration=45, avgTaskDuration=45.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791810894_0001_1_00_000000 ], longestDurationTasks=[ task_1452791810894_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:51,132 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791810894_0001_1_00 [BEF13210C18F42329A5C87C1676047D3] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:51,132 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791810894_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:51,133 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791810894_0001_1_01 [D562CB8D48274A37896EA06F4389CF7F] attempt: attempt_1452791810894_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:51,133 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: D562CB8D48274A37896EA06F4389CF7F
2016-01-14 17:16:51,133 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791810894_0001_1_01 [D562CB8D48274A37896EA06F4389CF7F]
2016-01-14 17:16:51,133 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791810894_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791810894_0001_00_000001, stoppedTime=1452791811133, exitStatus=0
2016-01-14 17:16:51,133 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791810894_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:51,133 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791810894_0001_1_00 [BEF13210C18F42329A5C87C1676047D3] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:51,133 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:51,133 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791810894_0001_1][Event:TASK_STARTED]: vertexName=D562CB8D48274A37896EA06F4389CF7F, taskId=task_1452791810894_0001_1_01_000000, scheduledTime=1452791811133, launchTime=1452791811133
2016-01-14 17:16:51,133 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791810894_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:51,134 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791810894_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:51,134 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:51,134 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=766509056, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791811122, Process_Duration=2, Process_End_Time=1452791811124, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:51,134 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:51,134 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:51,135 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791810894_0001_1_01 [D562CB8D48274A37896EA06F4389CF7F]
2016-01-14 17:16:51,135 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791810894_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:51,135 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791810894_0001_00_000001
2016-01-14 17:16:51,135 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791810894_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:51,136 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [14DF9C41F50948FB955D6B0EB3F9510E/CFAFB7C034A54E8397FCC05E2AE3B1C4] every/(1/1) ...emblyplatform/ALL_ARGS_-1, VertexName: D562CB8D48274A37896EA06F4389CF7F, VertexParallelism: 1, TaskAttemptID:attempt_1452791810894_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BEF13210C18F42329A5C87C1676047D3, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=F7EE78ED6EB443F794B0E45D5E1CED9C, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:51,136 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791810894_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:51,136 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:51,137 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791810894_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:51,137 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791810894_0001_00_000001, Exiting
2016-01-14 17:16:51,137 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791810894_0001_00_000001
2016-01-14 17:16:51,137 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791810894_0001_00_000001
2016-01-14 17:16:51,137 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791810894_0001_00_000002
2016-01-14 17:16:51,137 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791810894_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:51,139 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791810894_0001_00_000002
2016-01-14 17:16:51,140 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 17:16:51,141 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791810894_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:51,142 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791810894_0001_00_000001 completed successfully
2016-01-14 17:16:51,142 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791810894_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:51,142 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:51,142 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791810894_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:51,142 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791810894_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791810894_0001_00_000002, launchTime=1452791811142
2016-01-14 17:16:51,143 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791810894_0001_00_000002
2016-01-14 17:16:51,143 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791810894_0001_1_01_000000_0] to container: [container_1452791810894_0001_00_000002]
2016-01-14 17:16:51,143 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791810894_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:51,144 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791810894_0001_1_01_000000_0] started. Is using containerId: [container_1452791810894_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:51,144 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791810894_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D562CB8D48274A37896EA06F4389CF7F, taskAttemptId=attempt_1452791810894_0001_1_01_000000_0, startTime=1452791811144, containerId=container_1452791810894_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791810894_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:51,144 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791810894_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:51,144 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791810894_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:51,145 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:51,145 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791810894_0001_00_000002 given task: attempt_1452791810894_0001_1_01_000000_0
2016-01-14 17:16:51,145 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791810894_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791810894_0001_1_01_000000_0
2016-01-14 17:16:51 Starting to run new task attempt: attempt_1452791810894_0001_1_01_000000_0
2016-01-14 17:16:51,145 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791810894_0001_1_01_000000_0
2016-01-14 17:16:51,145 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:51,146 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:51,146 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:51,146 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:51,146 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:51,146 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:51,146 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [14DF9C41F50948FB955D6B0EB3F9510E/CFAFB7C034A54E8397FCC05E2AE3B1C4] every/(1/1) ...emblyplatform/ALL_ARGS_-1, VertexName: D562CB8D48274A37896EA06F4389CF7F, VertexParallelism: 1, TaskAttemptID:attempt_1452791810894_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BEF13210C18F42329A5C87C1676047D3, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=F7EE78ED6EB443F794B0E45D5E1CED9C, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:51,146 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=766509056
2016-01-14 17:16:51,149 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791810894_0001_1_01_000000_0
2016-01-14 17:16:51,149 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:51,149 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:51,149 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791810894_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:51,151 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=BEF13210C18F42329A5C87C1676047D3, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:51,151 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:51,152 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: BEF13210C18F42329A5C87C1676047D3
2016-01-14 17:16:51,153 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 689858112, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:51,154 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:51,154 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 689858112, type: INPUT, componentVertexName: BEF13210C18F42329A5C87C1676047D3
2016-01-14 17:16:51,154 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: BEF13210C18F42329A5C87C1676047D3
2016-01-14 17:16:51,155 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:51,158 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:51,159 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F7EE78ED6EB443F794B0E45D5E1CED9C, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:51,160 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:51,160 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F7EE78ED6EB443F794B0E45D5E1CED9C
2016-01-14 17:16:51,161 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:51,161 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: F7EE78ED6EB443F794B0E45D5E1CED9C
2016-01-14 17:16:51,165 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D562CB8D48274A37896EA06F4389CF7F, ordinal: 1
2016-01-14 17:16:51,168 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D562CB8D48274A37896EA06F4389CF7F, mem on start (mb), free: 519, total: 731, max: 731
2016-01-14 17:16:51,168 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:51,168 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:51,169 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:51,176 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:51,176 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F7EE78ED6EB443F794B0E45D5E1CED9C
2016-01-14 17:16:51,176 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:51,176 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:51,176 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:51,176 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:51,177 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:51,177 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:51,177 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 689858112, TotalRequestedScaled: 6.367921033846154E8, TotalJVMHeap: 766509056, TotalAvailable: 513561067, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:51,177 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, BEF13210C18F42329A5C87C1676047D3, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=689858112, allocated=513561067
2016-01-14 17:16:51,177 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F7EE78ED6EB443F794B0E45D5E1CED9C, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:51,177 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:51,177 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: BEF13210C18F42329A5C87C1676047D3 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:51,177 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:51,177 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:51,177 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: BEF13210C18F42329A5C87C1676047D3
2016-01-14 17:16:51,178 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:51,178 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:51,179 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: BEF13210C18F42329A5C87C1676047D3 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:51,179 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=689858112, postMergeMem=0, RuntimeTotalAvailable=513561067. Updated to: ShuffleMem=513561067, postMergeMem: 0
2016-01-14 17:16:51,179 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=513561067, maxSingleShuffleLimit=128390264, mergeThreshold=462204928, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:51,179 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:51,180 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: BEF13210C18F42329A5C87C1676047D3
2016-01-14 17:16:51,181 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:51,181 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791810894_0001_1_01_000000_0
2016-01-14 17:16:51,184 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:51,185 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 073D4B0EE49245D1897EBF487B243A77
2016-01-14 17:16:51,185 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_ARGS_-1"], id: F7EE78ED6EB443F794B0E45D5E1CED9C
2016-01-14 17:16:51,186 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_ARGS_-1"]
2016-01-14 17:16:51,186 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 073D4B0EE49245D1897EBF487B243A77, for 1 inputs
2016-01-14 17:16:51,187 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791810894_0001_1_00_000000_0_10321, runDuration: 33875, ]
2016-01-14 17:16:51,188 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791810894_0001_1_00_000000_0_10321] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791811188, TimeTaken=0, Rate=0.00 MB/s
2016-01-14 17:16:51,189 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : BEF13210C18F42329A5C87C1676047D3
2016-01-14 17:16:51,189 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:51,190 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [BEF13210C18F42329A5C87C1676047D3] #54 in 2ms
2016-01-14 17:16:51,190 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [BEF13210C18F42329A5C87C1676047D3] #54, status:false, isInterrupted:false
2016-01-14 17:16:51,191 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:51,191 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 17:16:51,191 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 17:16:51,191 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:51,191 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:51,191 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:51,191 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:51,191 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : BEF13210C18F42329A5C87C1676047D3
2016-01-14 17:16:51,192 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:51,192 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D562CB8D48274A37896EA06F4389CF7F, all 1 inputs ready in: 00:00:00.006
2016-01-14 17:16:51,193 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@3138c10; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@78507003
2016-01-14 17:16:51,193 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:51,193 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(92)) - caught throwable
cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:51,194 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:51,194 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:51,194 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791810894_0001_1_01_000000_0
2016-01-14 17:16:51,194 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791810894_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:51,195 INFO  task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1452791810894_0001_1_01_000000_0
cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:51,196 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=766509056, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=9, LAST_EVENT_RECEIVED=9][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791811192, Process_Duration=3, Process_End_Time=1452791811195, Read_Duration=0, Tuples_Read=1]]
2016-01-14 17:16:51,196 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D562CB8D48274A37896EA06F4389CF7F, mem on close (mb), free: 513, total: 731, max: 731
2016-01-14 17:16:51,196 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791810894_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D562CB8D48274A37896EA06F4389CF7F, taskAttemptId=attempt_1452791810894_0001_1_01_000000_0, startTime=1452791811144, finishTime=1452791811196, timeTaken=52, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=766509056, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=9, LAST_EVENT_RECEIVED=9, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791811192, Process_Duration=3, Process_End_Time=1452791811195, Read_Duration=0, Tuples_Read=1
2016-01-14 17:16:51,196 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791810894_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-01-14 17:16:51,196 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:51,196 INFO  impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1452791810894_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-01-14 17:16:51,197 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791810894_0001_1][Event:TASK_FINISHED]: vertexName=D562CB8D48274A37896EA06F4389CF7F, taskId=task_1452791810894_0001_1_01_000000, startTime=1452791811144, finishTime=1452791811196, timeTaken=52, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
], counters=Counters: 0
2016-01-14 17:16:51,197 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791810894_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-01-14 17:16:51,197 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:51,197 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791810894_0001_1_01 [D562CB8D48274A37896EA06F4389CF7F] : 1
2016-01-14 17:16:51,197 INFO  impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1452791810894_0001_1_01 [D562CB8D48274A37896EA06F4389CF7F] because task failed: task_1452791810894_0001_1_01_000000
2016-01-14 17:16:51,197 INFO  impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1452791810894_0001_1_01 [D562CB8D48274A37896EA06F4389CF7F] due to trigger: OWN_TASK_FAILURE
2016-01-14 17:16:51,197 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-01-14 17:16:51,197 INFO  impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1452791810894_0001_1_01 [D562CB8D48274A37896EA06F4389CF7F]
2016-01-14 17:16:51,197 INFO  impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=F7EE78ED6EB443F794B0E45D5E1CED9C, vertexId=vertex_1452791810894_0001_1_01 [D562CB8D48274A37896EA06F4389CF7F]
2016-01-14 17:16:51,198 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791810894_0001_1][Event:VERTEX_FINISHED]: vertexName=D562CB8D48274A37896EA06F4389CF7F, vertexId=vertex_1452791810894_0001_1_01, initRequestedTime=1452791811062, initedTime=1452791811069, startRequestedTime=1452791811077, startedTime=1452791811077, finishTime=1452791811198, timeTaken=121, status=FAILED, diagnostics=Task failed, taskId=task_1452791810894_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1452791810894_0001_1_01 [D562CB8D48274A37896EA06F4389CF7F] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1452791811144, firstTasksToStart=[ task_1452791810894_0001_1_01_000000 ], lastTaskFinishTime=1452791811196, lastTasksToFinish=[ task_1452791810894_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[  ], longestDurationTasks=[  ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-01-14 17:16:51,198 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:51,199 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:51,199 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=D562CB8D48274A37896EA06F4389CF7F, index=1
2016-01-14 17:16:51,199 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: BEF13210C18F42329A5C87C1676047D3
2016-01-14 17:16:51,199 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:51,199 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791810894_0001_1_01 [D562CB8D48274A37896EA06F4389CF7F] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-01-14 17:16:51,200 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:51,200 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791810894_0001_1_01 [D562CB8D48274A37896EA06F4389CF7F] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-01-14 17:16:51,200 INFO  impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-01-14 17:16:51,200 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791810894_0001_1
2016-01-14 17:16:51,200 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1452791810894_0001_1_00 [BEF13210C18F42329A5C87C1676047D3]
2016-01-14 17:16:51,200 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: F7EE78ED6EB443F794B0E45D5E1CED9C for vertex: vertex_1452791810894_0001_1_01 [D562CB8D48274A37896EA06F4389CF7F]
2016-01-14 17:16:51,200 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:51,200 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:51,200 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:51,200 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=D562CB8D48274A37896EA06F4389CF7F, sourceVertex=BEF13210C18F42329A5C87C1676047D3
2016-01-14 17:16:51,200 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=D562CB8D48274A37896EA06F4389CF7F, sourceVertex=F7EE78ED6EB443F794B0E45D5E1CED9C
2016-01-14 17:16:51,201 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791810894_0001_00_000002
2016-01-14 17:16:51,201 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791810894_0001_00_000002
2016-01-14 17:16:51,201 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791810894_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-01-14 17:16:51,201 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1452791810894_0001_00_000002 completed but with errors
2016-01-14 17:16:51,201 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791810894_0001_1, queueSize=0
2016-01-14 17:16:51,202 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:51,203 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791810894_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791810894_0001_1, startTime=1452791811060, finishTime=1452791811200, timeTaken=140, status=FAILED, diagnostics=Vertex failed, vertexName=D562CB8D48274A37896EA06F4389CF7F, vertexId=vertex_1452791810894_0001_1_01, diagnostics=[Task failed, taskId=task_1452791810894_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1452791810894_0001_1_01 [D562CB8D48274A37896EA06F4389CF7F] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=766509056, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791811122, Process_Duration=2, Process_End_Time=1452791811124, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:51,203 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791810894_0001_1 finished with state: FAILED
2016-01-14 17:16:51,203 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791810894_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:51,204 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:51,205 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791810894_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791810894_0001_00_000002, stoppedTime=1452791811203, exitStatus=0
2016-01-14 17:16:51,205 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791810894_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:51,206 INFO  node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1452791810894_0001_1_01_000000_0 failed: true container: container_1452791810894_0001_00_000002 numFailedTAs: 0
2016-01-14 17:16:51,207 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1452791810894_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-01-14 17:16:51,207 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791810894_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:16:51 Completed Dag: dag_1452791810894_0001_1
2016-01-14 17:16:51,207 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791810894_0001_1, dagState=FAILED
2016-01-14 17:16:51,207 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791810894_0001_1_post
2016-01-14 17:16:51,208 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:51,208 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:51,208 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452791810894_0001_00_000002
2016-01-14 17:16:51,208 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791810894_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-01-14 17:16:51,208 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...emblyplatform/ALL_ARGS_-1, applicationId=application_1452791810894_0001
2016-01-14 17:16:51,210 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:51,210 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:51,210 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:51,210 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:51,210 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:51,211 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:51,211 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:51,211 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:51,212 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:51,212 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:51,212 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:51,212 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:51,216 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:51,216 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/14DF9C41F50948FB955D6B0EB3F9510E/CFAFB7C034A54E8397FCC05E2AE3B1C4/.tez/application_1452791810894_0001
2016-01-14 17:16:51,217 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping all jobs
2016-01-14 17:16:51,217 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping: (1/1) ...emblyplatform/ALL_ARGS_-1
2016-01-14 17:16:51,217 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopped all jobs
2016-01-14 17:16:51,218 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.341
2016-01-14 17:16:51,218 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_ARGS_-1
2016-01-14 17:16:51,243 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:51,243 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:51,277 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.002
2016-01-14 17:16:51,279 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.005
2016-01-14 17:16:51,279 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 17:16:51,279 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 17:16:51,280 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_ARGS_'field'
2016-01-14 17:16:51,302 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:51,302 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:51,328 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.005
2016-01-14 17:16:51,336 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-01-14 17:16:51,336 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:51,336 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:51,336 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:51,358 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:51,362 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_ARGS_RESULTS
2016-01-14 17:16:51,402 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:51,402 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:51,402 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_ARGS_RESULTS"]
2016-01-14 17:16:51,402 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:51,402 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:51,402 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:51,404 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...platform/ALL_ARGS_RESULTS
2016-01-14 17:16:51,404 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:51,414 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:51,419 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:51,419 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:51,419 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:51,428 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/ED1ECACAF0B6491390F582B91745C624/EA1D9EDA6EFC4E73BBD9A549B32170C9/.tez/application_1452791811419_0001 doesn't exist and is created
2016-01-14 17:16:51,442 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:51,442 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:51,443 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/ED1ECACAF0B6491390F582B91745C624/EA1D9EDA6EFC4E73BBD9A549B32170C9/.tez/application_1452791811419_0001
2016-01-14 17:16:51,445 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791811419_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:51,454 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:51,454 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:51,455 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:51,455 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:51,455 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/ED1ECACAF0B6491390F582B91745C624/EA1D9EDA6EFC4E73BBD9A549B32170C9/.tez/application_1452791811419_0001/localmode-log-dir
2016-01-14 17:16:51,455 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/ED1ECACAF0B6491390F582B91745C624/EA1D9EDA6EFC4E73BBD9A549B32170C9/.tez/application_1452791811419_0001/localmode-log-dir/history.txt.appattempt_1452791811419_0001_000000, maxErrors=10
2016-01-14 17:16:51,456 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:51,456 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791811419_0001_000000, appSubmitTime=1452791811444, launchTime=1452791811444
2016-01-14 17:16:51,459 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:51,459 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:51,460 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:51,460 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:51,461 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:48050
2016-01-14 17:16:51,463 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:51,464 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791811419_0001_000000, startTime=1452791811464
2016-01-14 17:16:51,464 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:51,465 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:51,543 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:51,543 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:51,543 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...platform/ALL_ARGS_RESULTS, applicationId=application_1452791811419_0001, dagName=[ED1ECACAF0B6491390F582B91745C624/EA1D9EDA6EFC4E73BBD9A549B32170C9] every/(1/1) ...platform/ALL_ARGS_RESULTS
2016-01-14 17:16:51,558 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [ED1ECACAF0B6491390F582B91745C624/EA1D9EDA6EFC4E73BBD9A549B32170C9] every/(1/1) ...platform/ALL_ARGS_RESULTS
2016-01-14 17:16:51,559 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791811419_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/ED1ECACAF0B6491390F582B91745C624/EA1D9EDA6EFC4E73BBD9A549B32170C9/.tez/application_1452791811419_0001/localmode-log-dir/dag_1452791811419_0001_1.dot
2016-01-14 17:16:51,560 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/ED1ECACAF0B6491390F582B91745C624/EA1D9EDA6EFC4E73BBD9A549B32170C9/.tez/application_1452791811419_0001/localmode-log-dir/dag_1452791811419_0001_1-tez-dag.pb.txt
2016-01-14 17:16:51,565 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791811419_0001_1
2016-01-14 17:16:51,565 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:51,565 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [ED1ECACAF0B6491390F582B91745C624/EA1D9EDA6EFC4E73BBD9A549B32170C9] every/(1/1) ...platform/ALL_ARGS_RESULTS
2016-01-14 17:16:51 Running Dag: dag_1452791811419_0001_1
2016-01-14 17:16:51,576 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791811419_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791811419_0001_1, submitTime=1452791811558
2016-01-14 17:16:51,576 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:51,577 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex C3C0F468DC2847D69817D0E5A4CB4CD1
2016-01-14 17:16:51,577 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:51,577 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791811419_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791811419_0001_1, initTime=1452791811577
2016-01-14 17:16:51,577 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:51,578 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791811419_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:51,579 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791811419_0001_1][Event:DAG_STARTED]: dagID=dag_1452791811419_0001_1, startTime=1452791811579
2016-01-14 17:16:51,579 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:51,579 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...platform/ALL_ARGS_RESULTS, applicationId=application_1452791811419_0001, dagName=[ED1ECACAF0B6491390F582B91745C624/EA1D9EDA6EFC4E73BBD9A549B32170C9] every/(1/1) ...platform/ALL_ARGS_RESULTS
2016-01-14 17:16:51,579 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:51,579 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791811419_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:51,579 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: E9DEF9A38D6A4833935BDCC609896F73 : {7404F27A37C94035BE3202C8FB6AC6D9={InputName=7404F27A37C94035BE3202C8FB6AC6D9}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:51,579 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 7404F27A37C94035BE3202C8FB6AC6D9, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:51,579 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791811419_0001_1_00 [E9DEF9A38D6A4833935BDCC609896F73]
2016-01-14 17:16:51,579 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791811419_0001_1_00 [E9DEF9A38D6A4833935BDCC609896F73]
2016-01-14 17:16:51,580 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791811419_0001_1_00 [E9DEF9A38D6A4833935BDCC609896F73]
2016-01-14 17:16:51,580 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791811419_0001_1_00 [E9DEF9A38D6A4833935BDCC609896F73]. Starting root input initializers: 1
2016-01-14 17:16:51,580 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791811419_0001_1_00 [E9DEF9A38D6A4833935BDCC609896F73] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:51,580 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791811419_0001_1_01 [C3C0F468DC2847D69817D0E5A4CB4CD1]
2016-01-14 17:16:51,581 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 7404F27A37C94035BE3202C8FB6AC6D9 on vertex vertex_1452791811419_0001_1_00 [E9DEF9A38D6A4833935BDCC609896F73]
2016-01-14 17:16:51,583 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:51,583 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791811419_0001_1_01 [C3C0F468DC2847D69817D0E5A4CB4CD1]
2016-01-14 17:16:51,583 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791811419_0001_1_01 [C3C0F468DC2847D69817D0E5A4CB4CD1]
2016-01-14 17:16:51,583 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791811419_0001_1_01 [C3C0F468DC2847D69817D0E5A4CB4CD1]
2016-01-14 17:16:51,583 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=F17D8497D89449128F023D7105ACE96A, vertexId=vertex_1452791811419_0001_1_01 [C3C0F468DC2847D69817D0E5A4CB4CD1], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:51,584 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=F17D8497D89449128F023D7105ACE96A, vertexId=vertex_1452791811419_0001_1_01 [C3C0F468DC2847D69817D0E5A4CB4CD1]
2016-01-14 17:16:51,583 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:51,585 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=F17D8497D89449128F023D7105ACE96A, vertexName=C3C0F468DC2847D69817D0E5A4CB4CD1, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:51,585 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=F17D8497D89449128F023D7105ACE96A, vertexId=vertex_1452791811419_0001_1_01 [C3C0F468DC2847D69817D0E5A4CB4CD1]
2016-01-14 17:16:51,586 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 7404F27A37C94035BE3202C8FB6AC6D9 asking for 1 tasks. Headroom: 732 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:51,586 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:51,587 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:51,587 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791811419_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C3C0F468DC2847D69817D0E5A4CB4CD1, vertexId=vertex_1452791811419_0001_1_01, initRequestedTime=1452791811580, initedTime=1452791811587, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:51,587 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791811419_0001_1_01 [C3C0F468DC2847D69817D0E5A4CB4CD1] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:51,587 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:51,589 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:51,589 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 7404F27A37C94035BE3202C8FB6AC6D9 on vertex vertex_1452791811419_0001_1_00 [E9DEF9A38D6A4833935BDCC609896F73]
2016-01-14 17:16:51,589 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791811419_0001_1_00 [E9DEF9A38D6A4833935BDCC609896F73] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:51,590 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {7404F27A37C94035BE3202C8FB6AC6D9=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:51,590 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791811419_0001_1_00 [E9DEF9A38D6A4833935BDCC609896F73] parallelism set to 1
2016-01-14 17:16:51,590 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791811419_0001_1_00 [E9DEF9A38D6A4833935BDCC609896F73] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:51,591 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791811419_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791811419_0001_1_00, eventCount=1
2016-01-14 17:16:51,592 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791811419_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E9DEF9A38D6A4833935BDCC609896F73, vertexId=vertex_1452791811419_0001_1_00, initRequestedTime=1452791811579, initedTime=1452791811591, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:51,592 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791811419_0001_1_00 [E9DEF9A38D6A4833935BDCC609896F73] with distanceFromRoot: 0
2016-01-14 17:16:51,592 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:51,593 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791811419_0001_1_00 [E9DEF9A38D6A4833935BDCC609896F73] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:51,593 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in E9DEF9A38D6A4833935BDCC609896F73
2016-01-14 17:16:51,593 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791811419_0001_1_00 [E9DEF9A38D6A4833935BDCC609896F73]
2016-01-14 17:16:51,594 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:51,594 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791811419_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791811419_0001_1_00, startRequestedTime=1452791811593, startedTime=1452791811593
2016-01-14 17:16:51,595 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791811419_0001_1_00 [E9DEF9A38D6A4833935BDCC609896F73] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:51,595 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:51,596 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791811419_0001_1][Event:TASK_STARTED]: vertexName=E9DEF9A38D6A4833935BDCC609896F73, taskId=task_1452791811419_0001_1_00_000000, scheduledTime=1452791811595, launchTime=1452791811595
2016-01-14 17:16:51,596 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791811419_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:51,596 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791811419_0001_1_00 for vertex: vertex_1452791811419_0001_1_01 [C3C0F468DC2847D69817D0E5A4CB4CD1] numStartedSources: 1 numSources: 1
2016-01-14 17:16:51,597 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791811419_0001_1_01 [C3C0F468DC2847D69817D0E5A4CB4CD1] with distanceFromRoot: 1
2016-01-14 17:16:51,597 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791811419_0001, with dag id: dag_1452791811419_0001_1
2016-01-14 17:16:51,597 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791811419_0001_1_00 [E9DEF9A38D6A4833935BDCC609896F73]
2016-01-14 17:16:51,598 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: E9DEF9A38D6A4833935BDCC609896F73 in vertex: C3C0F468DC2847D69817D0E5A4CB4CD1
2016-01-14 17:16:51,598 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: C3C0F468DC2847D69817D0E5A4CB4CD1 with 1 source tasks and 1 pending tasks
2016-01-14 17:16:51,598 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:51,598 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791811419_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791811419_0001_1_01, startRequestedTime=1452791811597, startedTime=1452791811597
2016-01-14 17:16:51,599 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791811419_0001_1_01 [C3C0F468DC2847D69817D0E5A4CB4CD1] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:51,599 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [ED1ECACAF0B6491390F582B91745C624/EA1D9EDA6EFC4E73BBD9A549B32170C9] every/(1/1) ...platform/ALL_ARGS_RESULTS, VertexName: E9DEF9A38D6A4833935BDCC609896F73, VertexParallelism: 1, TaskAttemptID:attempt_1452791811419_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=7404F27A37C94035BE3202C8FB6AC6D9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C3C0F468DC2847D69817D0E5A4CB4CD1, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:51,599 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791811419_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:51,600 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:51,600 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:51,601 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:51,601 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:51,601 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791811419_0001_00_000001
2016-01-14 17:16:51,601 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791811419_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:51,603 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791811419_0001_00_000001
2016-01-14 17:16:51,603 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:51,604 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:51,604 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791811419_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:51,604 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791811419_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791811419_0001_00_000001, launchTime=1452791811604
2016-01-14 17:16:51,604 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791811419_0001_00_000001
2016-01-14 17:16:51,605 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791811419_0001_1_00_000000_0] to container: [container_1452791811419_0001_00_000001]
2016-01-14 17:16:51,605 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791811419_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:51,605 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791811419_0001_00_000001 given task: attempt_1452791811419_0001_1_00_000000_0
2016-01-14 17:16:51,605 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791811419_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791811419_0001_1_00_000000_0
2016-01-14 17:16:51,605 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791811419_0001_1_00_000000_0] started. Is using containerId: [container_1452791811419_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:51,605 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791811419_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E9DEF9A38D6A4833935BDCC609896F73, taskAttemptId=attempt_1452791811419_0001_1_00_000000_0, startTime=1452791811605, containerId=container_1452791811419_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791811419_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:51,605 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791811419_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:51 Starting to run new task attempt: attempt_1452791811419_0001_1_00_000000_0
2016-01-14 17:16:51,606 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:51,606 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791811419_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:51,606 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791811419_0001_1_00_000000_0
2016-01-14 17:16:51,606 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:51,606 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:51,606 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:51,606 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:51,606 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:51,606 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:51,606 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [ED1ECACAF0B6491390F582B91745C624/EA1D9EDA6EFC4E73BBD9A549B32170C9] every/(1/1) ...platform/ALL_ARGS_RESULTS, VertexName: E9DEF9A38D6A4833935BDCC609896F73, VertexParallelism: 1, TaskAttemptID:attempt_1452791811419_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=7404F27A37C94035BE3202C8FB6AC6D9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C3C0F468DC2847D69817D0E5A4CB4CD1, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:51,606 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=767557632
2016-01-14 17:16:51,607 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791811419_0001_1_00_000000_0
2016-01-14 17:16:51,608 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:51,608 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:51,608 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:51,608 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=7404F27A37C94035BE3202C8FB6AC6D9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:51,608 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:51,608 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 7404F27A37C94035BE3202C8FB6AC6D9
2016-01-14 17:16:51,609 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 7404F27A37C94035BE3202C8FB6AC6D9
2016-01-14 17:16:51,610 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:51,610 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:51,610 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 7404F27A37C94035BE3202C8FB6AC6D9
2016-01-14 17:16:51,610 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 7404F27A37C94035BE3202C8FB6AC6D9
2016-01-14 17:16:51,611 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=C3C0F468DC2847D69817D0E5A4CB4CD1, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:51,611 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:51,611 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: C3C0F468DC2847D69817D0E5A4CB4CD1
2016-01-14 17:16:51,612 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791811419_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:51,612 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E9DEF9A38D6A4833935BDCC609896F73, ordinal: 0
2016-01-14 17:16:51,612 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E9DEF9A38D6A4833935BDCC609896F73, mem on start (mb), free: 616, total: 732, max: 732
2016-01-14 17:16:51,612 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:51,613 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:51,613 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:51,613 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:51,613 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: C3C0F468DC2847D69817D0E5A4CB4CD1
2016-01-14 17:16:51,614 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: C3C0F468DC2847D69817D0E5A4CB4CD1
2016-01-14 17:16:51,614 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:51,614 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:51,614 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:51,614 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:51,614 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:51,614 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:51,615 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 767557632, TotalAvailable: 514263613, TotalRequested/TotalJVMHeap:0.14
2016-01-14 17:16:51,615 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 7404F27A37C94035BE3202C8FB6AC6D9, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:51,615 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, C3C0F468DC2847D69817D0E5A4CB4CD1, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:51,615 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:51,615 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 7404F27A37C94035BE3202C8FB6AC6D9
2016-01-14 17:16:51,615 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:51,615 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 7404F27A37C94035BE3202C8FB6AC6D9
2016-01-14 17:16:51,615 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 7404F27A37C94035BE3202C8FB6AC6D9 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:51,616 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:51,616 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:51,616 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:51,616 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791811419_0001_1_00_000000_0
2016-01-14 17:16:51,617 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:51,618 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@1a9aee92
2016-01-14 17:16:51,619 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:51,619 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@1a9aee92
2016-01-14 17:16:51,620 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:51,620 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:51,619 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 7404F27A37C94035BE3202C8FB6AC6D9
2016-01-14 17:16:51,620 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 3C4D4B1F286A49E8902E0DE835344211
2016-01-14 17:16:51,620 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 3C4D4B1F286A49E8902E0DE835344211
2016-01-14 17:16:51,621 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@7928b21c; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@5ec58a68; comparator=cascading.tuple.hadoop.util.TupleComparator@7b01e273; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:51,621 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:51,621 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:51,621 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:51,634 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:51,634 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:51,634 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:51,634 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:51,634 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:51,635 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:51,635 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:51,635 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E9DEF9A38D6A4833935BDCC609896F73, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:51,637 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791811419_0001_1_00_000000_0
2016-01-14 17:16:51,637 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E9DEF9A38D6A4833935BDCC609896F73, mem on close (mb), free: 513, total: 732, max: 732
2016-01-14 17:16:51,637 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:51,637 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:51,638 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:51,638 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:51,640 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:51 Completed running task attempt: attempt_1452791811419_0001_1_00_000000_0
2016-01-14 17:16:51,642 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791811419_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:51,643 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=767557632, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791811635, Process_Duration=2, Process_End_Time=1452791811637, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:51,643 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791811419_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791811419_0001_1_00, eventCount=1
2016-01-14 17:16:51,643 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:51,643 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:51,643 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:51,643 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791811419_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E9DEF9A38D6A4833935BDCC609896F73, taskAttemptId=attempt_1452791811419_0001_1_00_000000_0, startTime=1452791811605, finishTime=1452791811643, timeTaken=38, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=767557632, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791811635, Process_Duration=2, Process_End_Time=1452791811637, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:51,644 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791811419_0001_00_000001
2016-01-14 17:16:51,644 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791811419_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:51,644 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:51,644 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791811419_0001_1_00_000000_0
2016-01-14 17:16:51,644 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:51,644 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791811419_0001_1][Event:TASK_FINISHED]: vertexName=E9DEF9A38D6A4833935BDCC609896F73, taskId=task_1452791811419_0001_1_00_000000, startTime=1452791811605, finishTime=1452791811644, timeTaken=39, status=SUCCEEDED, successfulAttemptID=attempt_1452791811419_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=767557632, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791811635, Process_Duration=2, Process_End_Time=1452791811637, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:51,644 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791811419_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:51,645 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791811419_0001_1_00 [E9DEF9A38D6A4833935BDCC609896F73] : 1
2016-01-14 17:16:51,645 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791811419_0001_1_00 [E9DEF9A38D6A4833935BDCC609896F73]
2016-01-14 17:16:51,645 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:51,645 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791811419_0001_1][Event:VERTEX_FINISHED]: vertexName=E9DEF9A38D6A4833935BDCC609896F73, vertexId=vertex_1452791811419_0001_1_00, initRequestedTime=1452791811579, initedTime=1452791811591, startRequestedTime=1452791811593, startedTime=1452791811593, finishTime=1452791811645, timeTaken=52, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=767557632, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791811635, Process_Duration=2, Process_End_Time=1452791811637, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791811605, firstTasksToStart=[ task_1452791811419_0001_1_00_000000 ], lastTaskFinishTime=1452791811643, lastTasksToFinish=[ task_1452791811419_0001_1_00_000000 ], minTaskDuration=38, maxTaskDuration=38, avgTaskDuration=38.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791811419_0001_1_00_000000 ], longestDurationTasks=[ task_1452791811419_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:51,645 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791811419_0001_1_00 [E9DEF9A38D6A4833935BDCC609896F73] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:51,646 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791811419_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:51,646 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791811419_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791811419_0001_00_000001, stoppedTime=1452791811646, exitStatus=0
2016-01-14 17:16:51,646 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:51,646 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791811419_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:51,646 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791811419_0001_1_01 [C3C0F468DC2847D69817D0E5A4CB4CD1] attempt: attempt_1452791811419_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:51,646 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: C3C0F468DC2847D69817D0E5A4CB4CD1
2016-01-14 17:16:51,646 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791811419_0001_1_01 [C3C0F468DC2847D69817D0E5A4CB4CD1]
2016-01-14 17:16:51,646 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791811419_0001_1_00 [E9DEF9A38D6A4833935BDCC609896F73] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:51,647 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791811419_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:51,647 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791811419_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:51,647 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791811419_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:51,647 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791811419_0001_1][Event:TASK_STARTED]: vertexName=C3C0F468DC2847D69817D0E5A4CB4CD1, taskId=task_1452791811419_0001_1_01_000000, scheduledTime=1452791811647, launchTime=1452791811647
2016-01-14 17:16:51,647 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:51,647 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791811419_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:51,647 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791811419_0001_00_000001, Exiting
2016-01-14 17:16:51,648 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791811419_0001_00_000001
2016-01-14 17:16:51,648 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791811419_0001_00_000001
2016-01-14 17:16:51,648 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791811419_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:51,648 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791811419_0001_1_01 [C3C0F468DC2847D69817D0E5A4CB4CD1]
2016-01-14 17:16:51,648 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [ED1ECACAF0B6491390F582B91745C624/EA1D9EDA6EFC4E73BBD9A549B32170C9] every/(1/1) ...platform/ALL_ARGS_RESULTS, VertexName: C3C0F468DC2847D69817D0E5A4CB4CD1, VertexParallelism: 1, TaskAttemptID:attempt_1452791811419_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E9DEF9A38D6A4833935BDCC609896F73, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=F17D8497D89449128F023D7105ACE96A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:51,649 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791811419_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:51,649 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:51,649 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791811419_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:51,649 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791811419_0001_00_000001 completed successfully
2016-01-14 17:16:51,649 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791811419_0001_00_000002
2016-01-14 17:16:51,650 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791811419_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:51,651 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791811419_0001_00_000002
2016-01-14 17:16:51,651 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791811419_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:51,651 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 17:16:51,653 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791811419_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:51,653 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791811419_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791811419_0001_00_000002, launchTime=1452791811653
2016-01-14 17:16:51,653 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:51,654 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791811419_0001_00_000002
2016-01-14 17:16:51,655 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791811419_0001_1_01_000000_0] to container: [container_1452791811419_0001_00_000002]
2016-01-14 17:16:51,655 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791811419_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:51,655 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791811419_0001_1_01_000000_0] started. Is using containerId: [container_1452791811419_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:51,656 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791811419_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C3C0F468DC2847D69817D0E5A4CB4CD1, taskAttemptId=attempt_1452791811419_0001_1_01_000000_0, startTime=1452791811655, containerId=container_1452791811419_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791811419_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:51,656 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:51,656 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791811419_0001_00_000002 given task: attempt_1452791811419_0001_1_01_000000_0
2016-01-14 17:16:51,656 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791811419_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:51,656 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791811419_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791811419_0001_1_01_000000_0
2016-01-14 17:16:51,657 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791811419_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:51 Starting to run new task attempt: attempt_1452791811419_0001_1_01_000000_0
2016-01-14 17:16:51,657 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791811419_0001_1_01_000000_0
2016-01-14 17:16:51,657 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:51,657 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:51,657 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:51,658 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:51,658 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:51,658 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:51,658 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [ED1ECACAF0B6491390F582B91745C624/EA1D9EDA6EFC4E73BBD9A549B32170C9] every/(1/1) ...platform/ALL_ARGS_RESULTS, VertexName: C3C0F468DC2847D69817D0E5A4CB4CD1, VertexParallelism: 1, TaskAttemptID:attempt_1452791811419_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E9DEF9A38D6A4833935BDCC609896F73, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=F17D8497D89449128F023D7105ACE96A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:51,658 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=767557632
2016-01-14 17:16:51,659 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791811419_0001_1_01_000000_0
2016-01-14 17:16:51,659 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:51,659 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791811419_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:51,659 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:51,660 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=E9DEF9A38D6A4833935BDCC609896F73, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:51,660 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F17D8497D89449128F023D7105ACE96A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:51,660 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:51,660 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:51,661 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: E9DEF9A38D6A4833935BDCC609896F73
2016-01-14 17:16:51,661 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:51,661 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F17D8497D89449128F023D7105ACE96A
2016-01-14 17:16:51,661 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:51,661 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: F17D8497D89449128F023D7105ACE96A
2016-01-14 17:16:51,662 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 690801856, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:51,662 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:51,663 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 690801856, type: INPUT, componentVertexName: E9DEF9A38D6A4833935BDCC609896F73
2016-01-14 17:16:51,663 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: E9DEF9A38D6A4833935BDCC609896F73
2016-01-14 17:16:51,664 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:51,666 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C3C0F468DC2847D69817D0E5A4CB4CD1, ordinal: 1
2016-01-14 17:16:51,666 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C3C0F468DC2847D69817D0E5A4CB4CD1, mem on start (mb), free: 507, total: 732, max: 732
2016-01-14 17:16:51,666 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:51,666 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:51,666 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:51,671 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:51,672 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F17D8497D89449128F023D7105ACE96A
2016-01-14 17:16:51,672 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:51,672 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:51,672 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:51,672 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:51,672 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:51,672 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:51,672 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 690801856, TotalRequestedScaled: 6.376632516923077E8, TotalJVMHeap: 767557632, TotalAvailable: 514263613, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:51,673 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F17D8497D89449128F023D7105ACE96A, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:51,673 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, E9DEF9A38D6A4833935BDCC609896F73, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=690801856, allocated=514263613
2016-01-14 17:16:51,673 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:51,673 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: E9DEF9A38D6A4833935BDCC609896F73 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:51,673 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: E9DEF9A38D6A4833935BDCC609896F73
2016-01-14 17:16:51,673 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:51,673 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:51,673 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:51,673 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:51,674 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: E9DEF9A38D6A4833935BDCC609896F73 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:51,674 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=690801856, postMergeMem=0, RuntimeTotalAvailable=514263613. Updated to: ShuffleMem=514263613, postMergeMem: 0
2016-01-14 17:16:51,674 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=514263613, maxSingleShuffleLimit=128565904, mergeThreshold=462837248, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:51,674 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:51,675 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: E9DEF9A38D6A4833935BDCC609896F73
2016-01-14 17:16:51,675 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:51,675 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791811419_0001_1_01_000000_0
2016-01-14 17:16:51,676 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791811419_0001_1_00_000000_0_10327, runDuration: 29724, ]
2016-01-14 17:16:51,676 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791811419_0001_1_00_000000_0_10327] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791811676, TimeTaken=0, Rate=0.00 MB/s
2016-01-14 17:16:51,677 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : E9DEF9A38D6A4833935BDCC609896F73
2016-01-14 17:16:51,677 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:51,677 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [E9DEF9A38D6A4833935BDCC609896F73] #55 in 1ms
2016-01-14 17:16:51,677 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [E9DEF9A38D6A4833935BDCC609896F73] #55, status:false, isInterrupted:false
2016-01-14 17:16:51,677 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:51,678 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:51,678 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:51,678 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:51,678 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:51,678 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 3C4D4B1F286A49E8902E0DE835344211
2016-01-14 17:16:51,678 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:51,678 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_ARGS_RESULTS"], id: F17D8497D89449128F023D7105ACE96A
2016-01-14 17:16:51,678 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : E9DEF9A38D6A4833935BDCC609896F73
2016-01-14 17:16:51,678 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:51,679 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_ARGS_RESULTS"]
2016-01-14 17:16:51,679 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 3C4D4B1F286A49E8902E0DE835344211, for 1 inputs
2016-01-14 17:16:51,679 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C3C0F468DC2847D69817D0E5A4CB4CD1, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:51,679 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@7627621e; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@139adc2c
2016-01-14 17:16:51,680 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:51,682 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(92)) - caught throwable
cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:51,683 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:51,684 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:51,684 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791811419_0001_1_01_000000_0
2016-01-14 17:16:51,684 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791811419_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:51,684 INFO  task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1452791811419_0001_1_01_000000_0
cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:51,686 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=767557632, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=3, MERGE_PHASE_TIME=4, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791811679, Process_Duration=5, Process_End_Time=1452791811684, Read_Duration=1, Tuples_Read=1]]
2016-01-14 17:16:51,687 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C3C0F468DC2847D69817D0E5A4CB4CD1, mem on close (mb), free: 501, total: 732, max: 732
2016-01-14 17:16:51,687 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:51,687 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791811419_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C3C0F468DC2847D69817D0E5A4CB4CD1, taskAttemptId=attempt_1452791811419_0001_1_01_000000_0, startTime=1452791811655, finishTime=1452791811687, timeTaken=32, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=767557632, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=3, MERGE_PHASE_TIME=4, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791811679, Process_Duration=5, Process_End_Time=1452791811684, Read_Duration=1, Tuples_Read=1
2016-01-14 17:16:51,687 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791811419_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-01-14 17:16:51,688 INFO  impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1452791811419_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-01-14 17:16:51,688 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791811419_0001_1][Event:TASK_FINISHED]: vertexName=C3C0F468DC2847D69817D0E5A4CB4CD1, taskId=task_1452791811419_0001_1_01_000000, startTime=1452791811655, finishTime=1452791811688, timeTaken=33, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
], counters=Counters: 0
2016-01-14 17:16:51,688 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791811419_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-01-14 17:16:51,688 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791811419_0001_1_01 [C3C0F468DC2847D69817D0E5A4CB4CD1] : 1
2016-01-14 17:16:51,688 INFO  impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1452791811419_0001_1_01 [C3C0F468DC2847D69817D0E5A4CB4CD1] because task failed: task_1452791811419_0001_1_01_000000
2016-01-14 17:16:51,688 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:51,688 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:51,688 INFO  impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1452791811419_0001_1_01 [C3C0F468DC2847D69817D0E5A4CB4CD1] due to trigger: OWN_TASK_FAILURE
2016-01-14 17:16:51,689 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-01-14 17:16:51,689 INFO  impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1452791811419_0001_1_01 [C3C0F468DC2847D69817D0E5A4CB4CD1]
2016-01-14 17:16:51,688 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=C3C0F468DC2847D69817D0E5A4CB4CD1, index=1
2016-01-14 17:16:51,689 INFO  impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=F17D8497D89449128F023D7105ACE96A, vertexId=vertex_1452791811419_0001_1_01 [C3C0F468DC2847D69817D0E5A4CB4CD1]
2016-01-14 17:16:51,689 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: E9DEF9A38D6A4833935BDCC609896F73
2016-01-14 17:16:51,689 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:51,689 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:51,689 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:51,689 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:51,689 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:51,690 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=C3C0F468DC2847D69817D0E5A4CB4CD1, sourceVertex=E9DEF9A38D6A4833935BDCC609896F73
2016-01-14 17:16:51,690 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=C3C0F468DC2847D69817D0E5A4CB4CD1, sourceVertex=F17D8497D89449128F023D7105ACE96A
2016-01-14 17:16:51,690 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791811419_0001_00_000002
2016-01-14 17:16:51,690 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791811419_0001_00_000002
2016-01-14 17:16:51,690 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791811419_0001_1][Event:VERTEX_FINISHED]: vertexName=C3C0F468DC2847D69817D0E5A4CB4CD1, vertexId=vertex_1452791811419_0001_1_01, initRequestedTime=1452791811580, initedTime=1452791811587, startRequestedTime=1452791811597, startedTime=1452791811597, finishTime=1452791811690, timeTaken=93, status=FAILED, diagnostics=Task failed, taskId=task_1452791811419_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1452791811419_0001_1_01 [C3C0F468DC2847D69817D0E5A4CB4CD1] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1452791811655, firstTasksToStart=[ task_1452791811419_0001_1_01_000000 ], lastTaskFinishTime=1452791811687, lastTasksToFinish=[ task_1452791811419_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[  ], longestDurationTasks=[  ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-01-14 17:16:51,690 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791811419_0001_1_01 [C3C0F468DC2847D69817D0E5A4CB4CD1] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-01-14 17:16:51,690 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791811419_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-01-14 17:16:51,690 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1452791811419_0001_00_000002 completed but with errors
2016-01-14 17:16:51,690 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:51,691 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:51,690 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791811419_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791811419_0001_00_000002, stoppedTime=1452791811690, exitStatus=0
2016-01-14 17:16:51,691 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791811419_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:51,691 INFO  node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1452791811419_0001_1_01_000000_0 failed: true container: container_1452791811419_0001_00_000002 numFailedTAs: 0
2016-01-14 17:16:51,691 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791811419_0001_1_01 [C3C0F468DC2847D69817D0E5A4CB4CD1] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-01-14 17:16:51,691 INFO  impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-01-14 17:16:51,692 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791811419_0001_1
2016-01-14 17:16:51,692 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1452791811419_0001_1_00 [E9DEF9A38D6A4833935BDCC609896F73]
2016-01-14 17:16:51,692 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: F17D8497D89449128F023D7105ACE96A for vertex: vertex_1452791811419_0001_1_01 [C3C0F468DC2847D69817D0E5A4CB4CD1]
2016-01-14 17:16:51,692 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791811419_0001_1, queueSize=0
2016-01-14 17:16:51,694 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791811419_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791811419_0001_1, startTime=1452791811579, finishTime=1452791811692, timeTaken=113, status=FAILED, diagnostics=Vertex failed, vertexName=C3C0F468DC2847D69817D0E5A4CB4CD1, vertexId=vertex_1452791811419_0001_1_01, diagnostics=[Task failed, taskId=task_1452791811419_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1452791811419_0001_1_01 [C3C0F468DC2847D69817D0E5A4CB4CD1] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=767557632, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791811635, Process_Duration=2, Process_End_Time=1452791811637, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:51,695 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791811419_0001_1 finished with state: FAILED
2016-01-14 17:16:51,695 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791811419_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:51,697 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1452791811419_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-01-14 17:16:51,697 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:51,698 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791811419_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:16:51,698 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452791811419_0001_00_000002
2016-01-14 17:16:51 Completed Dag: dag_1452791811419_0001_1
2016-01-14 17:16:51,699 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791811419_0001_1, dagState=FAILED
2016-01-14 17:16:51,699 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791811419_0001_1_post
2016-01-14 17:16:51,699 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:51,700 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:51,700 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...platform/ALL_ARGS_RESULTS, applicationId=application_1452791811419_0001
2016-01-14 17:16:51,700 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791811419_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-01-14 17:16:51,702 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:51,702 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:51,702 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:51,702 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:51,702 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:51,703 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:51,703 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:51,703 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:51,703 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:51,703 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:51,704 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:51,704 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:51,704 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:51,708 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/ED1ECACAF0B6491390F582B91745C624/EA1D9EDA6EFC4E73BBD9A549B32170C9/.tez/application_1452791811419_0001
2016-01-14 17:16:51,708 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping all jobs
2016-01-14 17:16:51,708 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping: (1/1) ...platform/ALL_ARGS_RESULTS
2016-01-14 17:16:51,709 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopped all jobs
2016-01-14 17:16:51,709 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.307
2016-01-14 17:16:51,709 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_ARGS_RESULTS
2016-01-14 17:16:51,725 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 17:16:51,725 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 17:16:51,737 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:51,737 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:51,766 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.003
2016-01-14 17:16:51,767 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.004
2016-01-14 17:16:51,768 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 17:16:51,768 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 17:16:51,768 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_ARGS_ALL
2016-01-14 17:16:51,802 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:51,802 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:51,832 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.009
2016-01-14 17:16:51,836 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-01-14 17:16:51,836 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:51,836 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:51,837 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:51,855 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:51,861 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_ARGS_REPLACE
2016-01-14 17:16:51,910 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:51,911 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:51,911 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_ARGS_REPLACE"]
2016-01-14 17:16:51,911 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:51,911 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:51,911 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:51,911 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...platform/ALL_ARGS_REPLACE
2016-01-14 17:16:51,911 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:51,923 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:51,927 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:51,927 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:51,927 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:51,936 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/4709EDDB42004062A0705FC2133B04FC/BAD5D933FE884F5F99C66DB2353297BA/.tez/application_1452791811927_0001 doesn't exist and is created
2016-01-14 17:16:51,952 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:51,952 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:51,953 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/4709EDDB42004062A0705FC2133B04FC/BAD5D933FE884F5F99C66DB2353297BA/.tez/application_1452791811927_0001
2016-01-14 17:16:51,955 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791811927_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:51,964 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:51,965 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:51,966 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:51,966 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:51,966 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/4709EDDB42004062A0705FC2133B04FC/BAD5D933FE884F5F99C66DB2353297BA/.tez/application_1452791811927_0001/localmode-log-dir
2016-01-14 17:16:51,966 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/4709EDDB42004062A0705FC2133B04FC/BAD5D933FE884F5F99C66DB2353297BA/.tez/application_1452791811927_0001/localmode-log-dir/history.txt.appattempt_1452791811927_0001_000000, maxErrors=10
2016-01-14 17:16:51,966 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:51,966 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791811927_0001_000000, appSubmitTime=1452791811955, launchTime=1452791811955
2016-01-14 17:16:51,970 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:51,970 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:51,971 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:43660
2016-01-14 17:16:51,971 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:51,972 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:51,973 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:51,974 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791811927_0001_000000, startTime=1452791811974
2016-01-14 17:16:51,974 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:51,974 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:52,053 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:52,053 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:52,053 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...platform/ALL_ARGS_REPLACE, applicationId=application_1452791811927_0001, dagName=[4709EDDB42004062A0705FC2133B04FC/BAD5D933FE884F5F99C66DB2353297BA] every/(1/1) ...platform/ALL_ARGS_REPLACE
2016-01-14 17:16:52,067 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [4709EDDB42004062A0705FC2133B04FC/BAD5D933FE884F5F99C66DB2353297BA] every/(1/1) ...platform/ALL_ARGS_REPLACE
2016-01-14 17:16:52,068 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791811927_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/4709EDDB42004062A0705FC2133B04FC/BAD5D933FE884F5F99C66DB2353297BA/.tez/application_1452791811927_0001/localmode-log-dir/dag_1452791811927_0001_1.dot
2016-01-14 17:16:52,069 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/4709EDDB42004062A0705FC2133B04FC/BAD5D933FE884F5F99C66DB2353297BA/.tez/application_1452791811927_0001/localmode-log-dir/dag_1452791811927_0001_1-tez-dag.pb.txt
2016-01-14 17:16:52,074 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791811927_0001_1
2016-01-14 17:16:52,074 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:52,074 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [4709EDDB42004062A0705FC2133B04FC/BAD5D933FE884F5F99C66DB2353297BA] every/(1/1) ...platform/ALL_ARGS_REPLACE
2016-01-14 17:16:52 Running Dag: dag_1452791811927_0001_1
2016-01-14 17:16:52,086 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791811927_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791811927_0001_1, submitTime=1452791812067
2016-01-14 17:16:52,086 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:52,087 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex E58BF1702DD24744A7C6CC06F82D4CAA
2016-01-14 17:16:52,088 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:52,088 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791811927_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791811927_0001_1, initTime=1452791812087
2016-01-14 17:16:52,088 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791811927_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:52,088 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:52,088 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791811927_0001_1][Event:DAG_STARTED]: dagID=dag_1452791811927_0001_1, startTime=1452791812088
2016-01-14 17:16:52,088 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:52,088 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:52,089 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791811927_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:52,089 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: BCE0DB31094041D1BA3DEF6CB46F6DA5 : {AF31B0FBB8ED494FBC2CB28CCA837646={InputName=AF31B0FBB8ED494FBC2CB28CCA837646}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:52,089 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: AF31B0FBB8ED494FBC2CB28CCA837646, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:52,089 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791811927_0001_1_00 [BCE0DB31094041D1BA3DEF6CB46F6DA5]
2016-01-14 17:16:52,089 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791811927_0001_1_00 [BCE0DB31094041D1BA3DEF6CB46F6DA5]
2016-01-14 17:16:52,089 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791811927_0001_1_00 [BCE0DB31094041D1BA3DEF6CB46F6DA5]
2016-01-14 17:16:52,089 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...platform/ALL_ARGS_REPLACE, applicationId=application_1452791811927_0001, dagName=[4709EDDB42004062A0705FC2133B04FC/BAD5D933FE884F5F99C66DB2353297BA] every/(1/1) ...platform/ALL_ARGS_REPLACE
2016-01-14 17:16:52,089 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791811927_0001_1_00 [BCE0DB31094041D1BA3DEF6CB46F6DA5]. Starting root input initializers: 1
2016-01-14 17:16:52,090 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791811927_0001_1_00 [BCE0DB31094041D1BA3DEF6CB46F6DA5] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:52,090 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791811927_0001_1_01 [E58BF1702DD24744A7C6CC06F82D4CAA]
2016-01-14 17:16:52,092 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: AF31B0FBB8ED494FBC2CB28CCA837646 on vertex vertex_1452791811927_0001_1_00 [BCE0DB31094041D1BA3DEF6CB46F6DA5]
2016-01-14 17:16:52,092 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:52,093 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791811927_0001_1_01 [E58BF1702DD24744A7C6CC06F82D4CAA]
2016-01-14 17:16:52,093 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791811927_0001_1_01 [E58BF1702DD24744A7C6CC06F82D4CAA]
2016-01-14 17:16:52,093 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791811927_0001_1_01 [E58BF1702DD24744A7C6CC06F82D4CAA]
2016-01-14 17:16:52,093 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=203B89A2047D482B8502059025977FEF, vertexId=vertex_1452791811927_0001_1_01 [E58BF1702DD24744A7C6CC06F82D4CAA], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:52,093 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=203B89A2047D482B8502059025977FEF, vertexId=vertex_1452791811927_0001_1_01 [E58BF1702DD24744A7C6CC06F82D4CAA]
2016-01-14 17:16:52,095 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:52,095 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input AF31B0FBB8ED494FBC2CB28CCA837646 asking for 1 tasks. Headroom: 732 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:52,095 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:52,095 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=203B89A2047D482B8502059025977FEF, vertexName=E58BF1702DD24744A7C6CC06F82D4CAA, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:52,096 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=203B89A2047D482B8502059025977FEF, vertexId=vertex_1452791811927_0001_1_01 [E58BF1702DD24744A7C6CC06F82D4CAA]
2016-01-14 17:16:52,096 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:52,097 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:52,097 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791811927_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E58BF1702DD24744A7C6CC06F82D4CAA, vertexId=vertex_1452791811927_0001_1_01, initRequestedTime=1452791812090, initedTime=1452791812097, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:52,098 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791811927_0001_1_01 [E58BF1702DD24744A7C6CC06F82D4CAA] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:52,098 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:52,098 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: AF31B0FBB8ED494FBC2CB28CCA837646 on vertex vertex_1452791811927_0001_1_00 [BCE0DB31094041D1BA3DEF6CB46F6DA5]
2016-01-14 17:16:52,099 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791811927_0001_1_00 [BCE0DB31094041D1BA3DEF6CB46F6DA5] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:52,099 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {AF31B0FBB8ED494FBC2CB28CCA837646=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:52,099 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791811927_0001_1_00 [BCE0DB31094041D1BA3DEF6CB46F6DA5] parallelism set to 1
2016-01-14 17:16:52,099 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791811927_0001_1_00 [BCE0DB31094041D1BA3DEF6CB46F6DA5] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:52,100 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791811927_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791811927_0001_1_00, eventCount=1
2016-01-14 17:16:52,100 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791811927_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BCE0DB31094041D1BA3DEF6CB46F6DA5, vertexId=vertex_1452791811927_0001_1_00, initRequestedTime=1452791812089, initedTime=1452791812100, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:52,100 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791811927_0001_1_00 [BCE0DB31094041D1BA3DEF6CB46F6DA5] with distanceFromRoot: 0
2016-01-14 17:16:52,100 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:52,101 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791811927_0001_1_00 [BCE0DB31094041D1BA3DEF6CB46F6DA5] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:52,101 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in BCE0DB31094041D1BA3DEF6CB46F6DA5
2016-01-14 17:16:52,101 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791811927_0001_1_00 [BCE0DB31094041D1BA3DEF6CB46F6DA5]
2016-01-14 17:16:52,101 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:52,101 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791811927_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791811927_0001_1_00, startRequestedTime=1452791812101, startedTime=1452791812101
2016-01-14 17:16:52,102 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791811927_0001_1_00 [BCE0DB31094041D1BA3DEF6CB46F6DA5] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:52,102 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:52,102 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791811927_0001_1][Event:TASK_STARTED]: vertexName=BCE0DB31094041D1BA3DEF6CB46F6DA5, taskId=task_1452791811927_0001_1_00_000000, scheduledTime=1452791812102, launchTime=1452791812102
2016-01-14 17:16:52,102 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791811927_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:52,103 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791811927_0001_1_00 for vertex: vertex_1452791811927_0001_1_01 [E58BF1702DD24744A7C6CC06F82D4CAA] numStartedSources: 1 numSources: 1
2016-01-14 17:16:52,103 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791811927_0001_1_01 [E58BF1702DD24744A7C6CC06F82D4CAA] with distanceFromRoot: 1
2016-01-14 17:16:52,103 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791811927_0001_1_00 [BCE0DB31094041D1BA3DEF6CB46F6DA5]
2016-01-14 17:16:52,103 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: BCE0DB31094041D1BA3DEF6CB46F6DA5 in vertex: E58BF1702DD24744A7C6CC06F82D4CAA
2016-01-14 17:16:52,103 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: E58BF1702DD24744A7C6CC06F82D4CAA with 1 source tasks and 1 pending tasks
2016-01-14 17:16:52,104 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:52,104 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791811927_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791811927_0001_1_01, startRequestedTime=1452791812103, startedTime=1452791812103
2016-01-14 17:16:52,104 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791811927_0001_1_01 [E58BF1702DD24744A7C6CC06F82D4CAA] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:52,104 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [4709EDDB42004062A0705FC2133B04FC/BAD5D933FE884F5F99C66DB2353297BA] every/(1/1) ...platform/ALL_ARGS_REPLACE, VertexName: BCE0DB31094041D1BA3DEF6CB46F6DA5, VertexParallelism: 1, TaskAttemptID:attempt_1452791811927_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=AF31B0FBB8ED494FBC2CB28CCA837646, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E58BF1702DD24744A7C6CC06F82D4CAA, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:52,105 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791811927_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:52,105 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:52,105 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:52,106 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:52,106 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:52,106 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791811927_0001, with dag id: dag_1452791811927_0001_1
2016-01-14 17:16:52,106 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791811927_0001_00_000001
2016-01-14 17:16:52,106 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791811927_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:52,108 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791811927_0001_00_000001
2016-01-14 17:16:52,108 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:52,108 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791811927_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:52,109 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:52,109 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791811927_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791811927_0001_00_000001, launchTime=1452791812109
2016-01-14 17:16:52,110 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791811927_0001_00_000001
2016-01-14 17:16:52,111 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791811927_0001_1_00_000000_0] to container: [container_1452791811927_0001_00_000001]
2016-01-14 17:16:52,111 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791811927_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:52,112 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791811927_0001_1_00_000000_0] started. Is using containerId: [container_1452791811927_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:52,112 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791811927_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BCE0DB31094041D1BA3DEF6CB46F6DA5, taskAttemptId=attempt_1452791811927_0001_1_00_000000_0, startTime=1452791812112, containerId=container_1452791811927_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791811927_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:52,112 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791811927_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:52,112 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:52,112 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791811927_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:52,112 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791811927_0001_00_000001 given task: attempt_1452791811927_0001_1_00_000000_0
2016-01-14 17:16:52,112 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791811927_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791811927_0001_1_00_000000_0
2016-01-14 17:16:52 Starting to run new task attempt: attempt_1452791811927_0001_1_00_000000_0
2016-01-14 17:16:52,113 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791811927_0001_1_00_000000_0
2016-01-14 17:16:52,113 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:52,113 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:52,113 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:52,114 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:52,114 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:52,114 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:52,114 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [4709EDDB42004062A0705FC2133B04FC/BAD5D933FE884F5F99C66DB2353297BA] every/(1/1) ...platform/ALL_ARGS_REPLACE, VertexName: BCE0DB31094041D1BA3DEF6CB46F6DA5, VertexParallelism: 1, TaskAttemptID:attempt_1452791811927_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=AF31B0FBB8ED494FBC2CB28CCA837646, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E58BF1702DD24744A7C6CC06F82D4CAA, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:52,114 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=768081920
2016-01-14 17:16:52,115 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791811927_0001_1_00_000000_0
2016-01-14 17:16:52,115 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:52,115 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:52,116 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791811927_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:52,116 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=AF31B0FBB8ED494FBC2CB28CCA837646, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:52,116 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:52,117 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: AF31B0FBB8ED494FBC2CB28CCA837646
2016-01-14 17:16:52,117 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: AF31B0FBB8ED494FBC2CB28CCA837646
2016-01-14 17:16:52,118 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:52,118 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=E58BF1702DD24744A7C6CC06F82D4CAA, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:52,118 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:52,119 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: E58BF1702DD24744A7C6CC06F82D4CAA
2016-01-14 17:16:52,120 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:52,120 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: E58BF1702DD24744A7C6CC06F82D4CAA
2016-01-14 17:16:52,120 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:52,120 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: E58BF1702DD24744A7C6CC06F82D4CAA
2016-01-14 17:16:52,122 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:52,123 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: AF31B0FBB8ED494FBC2CB28CCA837646
2016-01-14 17:16:52,123 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: AF31B0FBB8ED494FBC2CB28CCA837646
2016-01-14 17:16:52,123 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BCE0DB31094041D1BA3DEF6CB46F6DA5, ordinal: 0
2016-01-14 17:16:52,123 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BCE0DB31094041D1BA3DEF6CB46F6DA5, mem on start (mb), free: 603, total: 732, max: 732
2016-01-14 17:16:52,124 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:52,124 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:52,124 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:52,124 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:52,124 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:52,124 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:52,124 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:52,124 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:52,125 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:52,125 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 768081920, TotalAvailable: 514614886, TotalRequested/TotalJVMHeap:0.14
2016-01-14 17:16:52,125 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, AF31B0FBB8ED494FBC2CB28CCA837646, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:52,125 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, E58BF1702DD24744A7C6CC06F82D4CAA, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:52,125 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:52,125 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: AF31B0FBB8ED494FBC2CB28CCA837646 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:52,125 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: AF31B0FBB8ED494FBC2CB28CCA837646
2016-01-14 17:16:52,126 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:52,126 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: AF31B0FBB8ED494FBC2CB28CCA837646
2016-01-14 17:16:52,126 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:52,126 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:52,126 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:52,126 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791811927_0001_1_00_000000_0
2016-01-14 17:16:52,128 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:52,129 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:52,129 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: AF31B0FBB8ED494FBC2CB28CCA837646
2016-01-14 17:16:52,129 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: FD789C5715384D18A097457CA1D671C1
2016-01-14 17:16:52,129 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] FD789C5715384D18A097457CA1D671C1
2016-01-14 17:16:52,129 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@124f8f49
2016-01-14 17:16:52,130 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@1ccb0514; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@54ba72d0; comparator=cascading.tuple.hadoop.util.TupleComparator@1cc7e125; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:52,130 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:52,130 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:52,130 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:52,130 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@124f8f49
2016-01-14 17:16:52,130 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:52,130 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:52,144 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:52,145 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:52,145 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:52,145 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:52,145 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:52,145 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:52,145 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:52,146 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BCE0DB31094041D1BA3DEF6CB46F6DA5, all 1 inputs ready in: 00:00:00.001
2016-01-14 17:16:52,147 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791811927_0001_1_00_000000_0
2016-01-14 17:16:52,148 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BCE0DB31094041D1BA3DEF6CB46F6DA5, mem on close (mb), free: 500, total: 732, max: 732
2016-01-14 17:16:52,148 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:52,148 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:52,148 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:52,148 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:52,151 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:52 Completed running task attempt: attempt_1452791811927_0001_1_00_000000_0
2016-01-14 17:16:52,154 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791811927_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:52,154 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791811927_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791811927_0001_1_00, eventCount=1
2016-01-14 17:16:52,154 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=768081920, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791812146, Process_Duration=1, Process_End_Time=1452791812147, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:52,155 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:52,155 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:52,155 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:52,155 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791811927_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BCE0DB31094041D1BA3DEF6CB46F6DA5, taskAttemptId=attempt_1452791811927_0001_1_00_000000_0, startTime=1452791812112, finishTime=1452791812154, timeTaken=42, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=768081920, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791812146, Process_Duration=1, Process_End_Time=1452791812147, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:52,155 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791811927_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:52,155 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791811927_0001_00_000001
2016-01-14 17:16:52,155 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:52,155 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791811927_0001_1_00_000000_0
2016-01-14 17:16:52,156 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:52,156 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791811927_0001_1][Event:TASK_FINISHED]: vertexName=BCE0DB31094041D1BA3DEF6CB46F6DA5, taskId=task_1452791811927_0001_1_00_000000, startTime=1452791812112, finishTime=1452791812155, timeTaken=43, status=SUCCEEDED, successfulAttemptID=attempt_1452791811927_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=768081920, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791812146, Process_Duration=1, Process_End_Time=1452791812147, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:52,156 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791811927_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:52,156 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791811927_0001_1_00 [BCE0DB31094041D1BA3DEF6CB46F6DA5] : 1
2016-01-14 17:16:52,156 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791811927_0001_1_00 [BCE0DB31094041D1BA3DEF6CB46F6DA5]
2016-01-14 17:16:52,156 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:52,156 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791811927_0001_1][Event:VERTEX_FINISHED]: vertexName=BCE0DB31094041D1BA3DEF6CB46F6DA5, vertexId=vertex_1452791811927_0001_1_00, initRequestedTime=1452791812089, initedTime=1452791812100, startRequestedTime=1452791812101, startedTime=1452791812101, finishTime=1452791812156, timeTaken=55, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=768081920, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791812146, Process_Duration=1, Process_End_Time=1452791812147, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791812112, firstTasksToStart=[ task_1452791811927_0001_1_00_000000 ], lastTaskFinishTime=1452791812154, lastTasksToFinish=[ task_1452791811927_0001_1_00_000000 ], minTaskDuration=42, maxTaskDuration=42, avgTaskDuration=42.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791811927_0001_1_00_000000 ], longestDurationTasks=[ task_1452791811927_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:52,157 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791811927_0001_1_00 [BCE0DB31094041D1BA3DEF6CB46F6DA5] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:52,157 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791811927_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:52,157 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791811927_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791811927_0001_00_000001, stoppedTime=1452791812157, exitStatus=0
2016-01-14 17:16:52,157 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:52,157 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791811927_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:52,157 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791811927_0001_1_01 [E58BF1702DD24744A7C6CC06F82D4CAA] attempt: attempt_1452791811927_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:52,157 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: E58BF1702DD24744A7C6CC06F82D4CAA
2016-01-14 17:16:52,157 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791811927_0001_1_01 [E58BF1702DD24744A7C6CC06F82D4CAA]
2016-01-14 17:16:52,158 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791811927_0001_1_00 [BCE0DB31094041D1BA3DEF6CB46F6DA5] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:52,158 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791811927_0001_1][Event:TASK_STARTED]: vertexName=E58BF1702DD24744A7C6CC06F82D4CAA, taskId=task_1452791811927_0001_1_01_000000, scheduledTime=1452791812158, launchTime=1452791812158
2016-01-14 17:16:52,158 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791811927_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:52,158 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791811927_0001_1_01 [E58BF1702DD24744A7C6CC06F82D4CAA]
2016-01-14 17:16:52,158 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [4709EDDB42004062A0705FC2133B04FC/BAD5D933FE884F5F99C66DB2353297BA] every/(1/1) ...platform/ALL_ARGS_REPLACE, VertexName: E58BF1702DD24744A7C6CC06F82D4CAA, VertexParallelism: 1, TaskAttemptID:attempt_1452791811927_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BCE0DB31094041D1BA3DEF6CB46F6DA5, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=203B89A2047D482B8502059025977FEF, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:52,158 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791811927_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:52,159 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:52,159 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:52,159 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791811927_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:52,159 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791811927_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:52,159 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791811927_0001_00_000002
2016-01-14 17:16:52,159 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791811927_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:52,161 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791811927_0001_00_000002
2016-01-14 17:16:52,161 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:52,162 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:52,162 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791811927_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791811927_0001_00_000002, launchTime=1452791812162
2016-01-14 17:16:52,162 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791811927_0001_00_000002
2016-01-14 17:16:52,162 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791811927_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:52,163 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791811927_0001_1_01_000000_0] to container: [container_1452791811927_0001_00_000002]
2016-01-14 17:16:52,163 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791811927_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:52,163 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791811927_0001_00_000002 given task: attempt_1452791811927_0001_1_01_000000_0
2016-01-14 17:16:52,163 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791811927_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791811927_0001_1_01_000000_0
2016-01-14 17:16:52,164 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791811927_0001_1_01_000000_0] started. Is using containerId: [container_1452791811927_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:52,164 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791811927_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E58BF1702DD24744A7C6CC06F82D4CAA, taskAttemptId=attempt_1452791811927_0001_1_01_000000_0, startTime=1452791812163, containerId=container_1452791811927_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791811927_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:52 Starting to run new task attempt: attempt_1452791811927_0001_1_01_000000_0
2016-01-14 17:16:52,164 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:52,164 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791811927_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:52,164 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791811927_0001_1_01_000000_0
2016-01-14 17:16:52,164 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:52,164 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791811927_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:52,164 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:52,164 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:52,164 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:52,165 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:52,165 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:52,165 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [4709EDDB42004062A0705FC2133B04FC/BAD5D933FE884F5F99C66DB2353297BA] every/(1/1) ...platform/ALL_ARGS_REPLACE, VertexName: E58BF1702DD24744A7C6CC06F82D4CAA, VertexParallelism: 1, TaskAttemptID:attempt_1452791811927_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BCE0DB31094041D1BA3DEF6CB46F6DA5, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=203B89A2047D482B8502059025977FEF, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:52,165 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=768081920
2016-01-14 17:16:52,165 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791811927_0001_1_01_000000_0
2016-01-14 17:16:52,165 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:52,166 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:52,166 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791811927_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:52,166 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791811927_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:52,166 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791811927_0001_00_000001, Exiting
2016-01-14 17:16:52,166 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791811927_0001_00_000001
2016-01-14 17:16:52,166 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791811927_0001_00_000001
2016-01-14 17:16:52,167 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791811927_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:52,167 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:52,167 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=BCE0DB31094041D1BA3DEF6CB46F6DA5, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:52,167 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:52,167 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: BCE0DB31094041D1BA3DEF6CB46F6DA5
2016-01-14 17:16:52,169 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 691273728, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:52,169 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:52,169 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 691273728, type: INPUT, componentVertexName: BCE0DB31094041D1BA3DEF6CB46F6DA5
2016-01-14 17:16:52,169 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: BCE0DB31094041D1BA3DEF6CB46F6DA5
2016-01-14 17:16:52,169 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=203B89A2047D482B8502059025977FEF, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:52,169 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:52,170 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:52,170 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 203B89A2047D482B8502059025977FEF
2016-01-14 17:16:52,170 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:52,170 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 203B89A2047D482B8502059025977FEF
2016-01-14 17:16:52,171 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791811927_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:52,171 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791811927_0001_00_000001 completed successfully
2016-01-14 17:16:52,171 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791811927_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:52,173 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E58BF1702DD24744A7C6CC06F82D4CAA, ordinal: 1
2016-01-14 17:16:52,173 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E58BF1702DD24744A7C6CC06F82D4CAA, mem on start (mb), free: 493, total: 732, max: 732
2016-01-14 17:16:52,173 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:52,173 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:52,174 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:52,181 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:52,181 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 203B89A2047D482B8502059025977FEF
2016-01-14 17:16:52,181 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:52,182 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:52,182 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:52,182 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:52,182 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:52,182 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:52,182 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 691273728, TotalRequestedScaled: 6.380988258461539E8, TotalJVMHeap: 768081920, TotalAvailable: 514614886, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:52,182 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, BCE0DB31094041D1BA3DEF6CB46F6DA5, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=691273728, allocated=514614886
2016-01-14 17:16:52,182 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 203B89A2047D482B8502059025977FEF, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:52,182 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:52,182 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: BCE0DB31094041D1BA3DEF6CB46F6DA5 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:52,183 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:52,183 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:52,183 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: BCE0DB31094041D1BA3DEF6CB46F6DA5
2016-01-14 17:16:52,183 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:52,183 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:52,184 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: BCE0DB31094041D1BA3DEF6CB46F6DA5 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:52,184 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=691273728, postMergeMem=0, RuntimeTotalAvailable=514614886. Updated to: ShuffleMem=514614886, postMergeMem: 0
2016-01-14 17:16:52,184 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=514614886, maxSingleShuffleLimit=128653720, mergeThreshold=463153376, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:52,184 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:52,184 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: BCE0DB31094041D1BA3DEF6CB46F6DA5
2016-01-14 17:16:52,184 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:52,185 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791811927_0001_1_01_000000_0
2016-01-14 17:16:52,186 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791811927_0001_1_00_000000_0_10333, runDuration: 34233, ]
2016-01-14 17:16:52,187 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791811927_0001_1_00_000000_0_10333] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791812187, TimeTaken=0, Rate=0.00 MB/s
2016-01-14 17:16:52,187 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : BCE0DB31094041D1BA3DEF6CB46F6DA5
2016-01-14 17:16:52,188 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:52,188 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:52,188 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [BCE0DB31094041D1BA3DEF6CB46F6DA5] #56 in 1ms
2016-01-14 17:16:52,188 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: FD789C5715384D18A097457CA1D671C1
2016-01-14 17:16:52,188 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [BCE0DB31094041D1BA3DEF6CB46F6DA5] #56, status:false, isInterrupted:false
2016-01-14 17:16:52,188 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:52,189 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:52,189 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:52,189 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:52,189 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:52,189 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : BCE0DB31094041D1BA3DEF6CB46F6DA5
2016-01-14 17:16:52,189 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:52,189 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_ARGS_REPLACE"], id: 203B89A2047D482B8502059025977FEF
2016-01-14 17:16:52,190 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_ARGS_REPLACE"]
2016-01-14 17:16:52,190 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] FD789C5715384D18A097457CA1D671C1, for 1 inputs
2016-01-14 17:16:52,190 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E58BF1702DD24744A7C6CC06F82D4CAA, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:52,190 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@7350eaef; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@45702c0e
2016-01-14 17:16:52,191 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:52,191 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(92)) - caught throwable
cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:52,192 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:52,193 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:52,193 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791811927_0001_1_01_000000_0
2016-01-14 17:16:52,193 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791811927_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:52,194 INFO  task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1452791811927_0001_1_01_000000_0
cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:52,196 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=768081920, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791812190, Process_Duration=4, Process_End_Time=1452791812194, Read_Duration=0, Tuples_Read=1]]
2016-01-14 17:16:52,196 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E58BF1702DD24744A7C6CC06F82D4CAA, mem on close (mb), free: 488, total: 732, max: 732
2016-01-14 17:16:52,196 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791811927_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E58BF1702DD24744A7C6CC06F82D4CAA, taskAttemptId=attempt_1452791811927_0001_1_01_000000_0, startTime=1452791812163, finishTime=1452791812196, timeTaken=33, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=768081920, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791812190, Process_Duration=4, Process_End_Time=1452791812194, Read_Duration=0, Tuples_Read=1
2016-01-14 17:16:52,196 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:52,197 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=E58BF1702DD24744A7C6CC06F82D4CAA, index=1
2016-01-14 17:16:52,197 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: BCE0DB31094041D1BA3DEF6CB46F6DA5
2016-01-14 17:16:52,197 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:52,197 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:52,197 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:52,198 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:52,198 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:52,196 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791811927_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-01-14 17:16:52,198 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=E58BF1702DD24744A7C6CC06F82D4CAA, sourceVertex=BCE0DB31094041D1BA3DEF6CB46F6DA5
2016-01-14 17:16:52,198 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=E58BF1702DD24744A7C6CC06F82D4CAA, sourceVertex=203B89A2047D482B8502059025977FEF
2016-01-14 17:16:52,198 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791811927_0001_00_000002
2016-01-14 17:16:52,199 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791811927_0001_00_000002
2016-01-14 17:16:52,198 INFO  impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1452791811927_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-01-14 17:16:52,199 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791811927_0001_1][Event:TASK_FINISHED]: vertexName=E58BF1702DD24744A7C6CC06F82D4CAA, taskId=task_1452791811927_0001_1_01_000000, startTime=1452791812163, finishTime=1452791812199, timeTaken=36, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
], counters=Counters: 0
2016-01-14 17:16:52,199 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791811927_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-01-14 17:16:52,199 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791811927_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-01-14 17:16:52,199 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1452791811927_0001_00_000002 completed but with errors
2016-01-14 17:16:52,199 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:52,200 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:52,200 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791811927_0001_1_01 [E58BF1702DD24744A7C6CC06F82D4CAA] : 1
2016-01-14 17:16:52,200 INFO  impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1452791811927_0001_1_01 [E58BF1702DD24744A7C6CC06F82D4CAA] because task failed: task_1452791811927_0001_1_01_000000
2016-01-14 17:16:52,200 INFO  impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1452791811927_0001_1_01 [E58BF1702DD24744A7C6CC06F82D4CAA] due to trigger: OWN_TASK_FAILURE
2016-01-14 17:16:52,200 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-01-14 17:16:52,200 INFO  impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1452791811927_0001_1_01 [E58BF1702DD24744A7C6CC06F82D4CAA]
2016-01-14 17:16:52,200 INFO  impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=203B89A2047D482B8502059025977FEF, vertexId=vertex_1452791811927_0001_1_01 [E58BF1702DD24744A7C6CC06F82D4CAA]
2016-01-14 17:16:52,201 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791811927_0001_1][Event:VERTEX_FINISHED]: vertexName=E58BF1702DD24744A7C6CC06F82D4CAA, vertexId=vertex_1452791811927_0001_1_01, initRequestedTime=1452791812090, initedTime=1452791812097, startRequestedTime=1452791812103, startedTime=1452791812103, finishTime=1452791812201, timeTaken=98, status=FAILED, diagnostics=Task failed, taskId=task_1452791811927_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1452791811927_0001_1_01 [E58BF1702DD24744A7C6CC06F82D4CAA] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1452791812163, firstTasksToStart=[ task_1452791811927_0001_1_01_000000 ], lastTaskFinishTime=1452791812196, lastTasksToFinish=[ task_1452791811927_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[  ], longestDurationTasks=[  ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-01-14 17:16:52,202 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791811927_0001_1_01 [E58BF1702DD24744A7C6CC06F82D4CAA] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-01-14 17:16:52,202 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(686)) - Container container_1452791811927_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-01-14 17:16:52,202 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791811927_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791811927_0001_00_000002, stoppedTime=1452791812202, exitStatus=1
2016-01-14 17:16:52,202 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791811927_0001_00_000002 transitioned from RUNNING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:52,202 INFO  node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1452791811927_0001_1_01_000000_0 failed: true container: container_1452791811927_0001_00_000002 numFailedTAs: 0
2016-01-14 17:16:52,201 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:52,203 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:52,203 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791811927_0001_1_01 [E58BF1702DD24744A7C6CC06F82D4CAA] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-01-14 17:16:52,203 INFO  impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-01-14 17:16:52,203 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791811927_0001_1
2016-01-14 17:16:52,203 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 203B89A2047D482B8502059025977FEF for vertex: vertex_1452791811927_0001_1_01 [E58BF1702DD24744A7C6CC06F82D4CAA]
2016-01-14 17:16:52,203 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1452791811927_0001_1_00 [BCE0DB31094041D1BA3DEF6CB46F6DA5]
2016-01-14 17:16:52,205 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791811927_0001_1, queueSize=0
2016-01-14 17:16:52,205 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:52,205 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791811927_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791811927_0001_1, startTime=1452791812088, finishTime=1452791812204, timeTaken=116, status=FAILED, diagnostics=Vertex failed, vertexName=E58BF1702DD24744A7C6CC06F82D4CAA, vertexId=vertex_1452791811927_0001_1_01, diagnostics=[Task failed, taskId=task_1452791811927_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1452791811927_0001_1_01 [E58BF1702DD24744A7C6CC06F82D4CAA] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=768081920, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791812146, Process_Duration=1, Process_End_Time=1452791812147, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:52,205 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791811927_0001_1 finished with state: FAILED
2016-01-14 17:16:52,205 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791811927_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:52,206 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791811927_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-01-14 17:16:52 Completed Dag: dag_1452791811927_0001_1
2016-01-14 17:16:52,206 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791811927_0001_1, dagState=FAILED
2016-01-14 17:16:52,206 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791811927_0001_1_post
2016-01-14 17:16:52,206 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:52,206 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:52,215 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...platform/ALL_ARGS_REPLACE, applicationId=application_1452791811927_0001
2016-01-14 17:16:52,216 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:52,217 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:52,217 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:52,217 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:52,217 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:52,217 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:52,217 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:52,217 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:52,218 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:52,218 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:52,218 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:52,219 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:52,219 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:52,223 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/4709EDDB42004062A0705FC2133B04FC/BAD5D933FE884F5F99C66DB2353297BA/.tez/application_1452791811927_0001
2016-01-14 17:16:52,224 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping all jobs
2016-01-14 17:16:52,224 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping: (1/1) ...platform/ALL_ARGS_REPLACE
2016-01-14 17:16:52,225 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopped all jobs
2016-01-14 17:16:52,225 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.315
2016-01-14 17:16:52,225 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_ARGS_REPLACE
2016-01-14 17:16:52,230 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 17:16:52,230 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 17:16:52,244 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:16:52,244 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:16:52,286 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.008
2016-01-14 17:16:52,292 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-01-14 17:16:52,293 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:52,293 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:16:52,293 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:16:52,305 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 17:16:52,309 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_ARGS_SWAP
2016-01-14 17:16:52,351 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 17:16:52,351 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:52,351 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_ARGS_SWAP"]
2016-01-14 17:16:52,351 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  parallel execution of steps is enabled: false
2016-01-14 17:16:52,351 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  executing total steps: 1
2016-01-14 17:16:52,351 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  allocating management threads: 1
2016-01-14 17:16:52,352 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...blyplatform/ALL_ARGS_SWAP
2016-01-14 17:16:52,352 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:16:52,363 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 17:16:52,368 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:52,368 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:16:52,368 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 17:16:52,377 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/54315A4EF3A34510B19F1F2E4BB9ECF2/E32AD6A461244110B633456F51F0A6D3/.tez/application_1452791812368_0001 doesn't exist and is created
2016-01-14 17:16:52,392 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:16:52,392 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:16:52,393 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/54315A4EF3A34510B19F1F2E4BB9ECF2/E32AD6A461244110B633456F51F0A6D3/.tez/application_1452791812368_0001
2016-01-14 17:16:52,396 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452791812368_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:16:52,405 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:16:52,405 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:16:52,407 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:16:52,407 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:16:52,407 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/54315A4EF3A34510B19F1F2E4BB9ECF2/E32AD6A461244110B633456F51F0A6D3/.tez/application_1452791812368_0001/localmode-log-dir
2016-01-14 17:16:52,407 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/54315A4EF3A34510B19F1F2E4BB9ECF2/E32AD6A461244110B633456F51F0A6D3/.tez/application_1452791812368_0001/localmode-log-dir/history.txt.appattempt_1452791812368_0001_000000, maxErrors=10
2016-01-14 17:16:52,407 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:16:52,407 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452791812368_0001_000000, appSubmitTime=1452791812395, launchTime=1452791812395
2016-01-14 17:16:52,409 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:16:52,410 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:16:52,411 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:16:52,412 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:16:52,414 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:59433
2016-01-14 17:16:52,415 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:16:52,417 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452791812368_0001_000000, startTime=1452791812417
2016-01-14 17:16:52,417 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:16:52,417 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:16:52,493 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:16:52,493 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:16:52,493 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...blyplatform/ALL_ARGS_SWAP, applicationId=application_1452791812368_0001, dagName=[54315A4EF3A34510B19F1F2E4BB9ECF2/E32AD6A461244110B633456F51F0A6D3] every/(1/1) ...blyplatform/ALL_ARGS_SWAP
2016-01-14 17:16:52,507 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [54315A4EF3A34510B19F1F2E4BB9ECF2/E32AD6A461244110B633456F51F0A6D3] every/(1/1) ...blyplatform/ALL_ARGS_SWAP
2016-01-14 17:16:52,508 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452791812368_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/54315A4EF3A34510B19F1F2E4BB9ECF2/E32AD6A461244110B633456F51F0A6D3/.tez/application_1452791812368_0001/localmode-log-dir/dag_1452791812368_0001_1.dot
2016-01-14 17:16:52,508 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/54315A4EF3A34510B19F1F2E4BB9ECF2/E32AD6A461244110B633456F51F0A6D3/.tez/application_1452791812368_0001/localmode-log-dir/dag_1452791812368_0001_1-tez-dag.pb.txt
2016-01-14 17:16:52,513 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791812368_0001_1
2016-01-14 17:16:52,514 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:52,514 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [54315A4EF3A34510B19F1F2E4BB9ECF2/E32AD6A461244110B633456F51F0A6D3] every/(1/1) ...blyplatform/ALL_ARGS_SWAP
2016-01-14 17:16:52 Running Dag: dag_1452791812368_0001_1
2016-01-14 17:16:52,525 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791812368_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452791812368_0001_1, submitTime=1452791812507
2016-01-14 17:16:52,525 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:16:52,525 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex BDD34027789442F69585C8BC8CFE6765
2016-01-14 17:16:52,525 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:16:52,526 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791812368_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452791812368_0001_1, initTime=1452791812525
2016-01-14 17:16:52,526 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791812368_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:16:52,526 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:16:52,526 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791812368_0001_1][Event:DAG_STARTED]: dagID=dag_1452791812368_0001_1, startTime=1452791812526
2016-01-14 17:16:52,526 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:16:52,526 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:16:52,526 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791812368_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:16:52,527 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: A9D84B543AA74502BCE2039A7E3D8AA9 : {62EA027B082E4244B7055B0990C97447={InputName=62EA027B082E4244B7055B0990C97447}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:16:52,527 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 62EA027B082E4244B7055B0990C97447, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:16:52,527 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452791812368_0001_1_00 [A9D84B543AA74502BCE2039A7E3D8AA9]
2016-01-14 17:16:52,527 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452791812368_0001_1_00 [A9D84B543AA74502BCE2039A7E3D8AA9]
2016-01-14 17:16:52,527 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452791812368_0001_1_00 [A9D84B543AA74502BCE2039A7E3D8AA9]
2016-01-14 17:16:52,527 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452791812368_0001_1_00 [A9D84B543AA74502BCE2039A7E3D8AA9]. Starting root input initializers: 1
2016-01-14 17:16:52,528 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791812368_0001_1_00 [A9D84B543AA74502BCE2039A7E3D8AA9] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:16:52,528 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452791812368_0001_1_01 [BDD34027789442F69585C8BC8CFE6765]
2016-01-14 17:16:52,528 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...blyplatform/ALL_ARGS_SWAP, applicationId=application_1452791812368_0001, dagName=[54315A4EF3A34510B19F1F2E4BB9ECF2/E32AD6A461244110B633456F51F0A6D3] every/(1/1) ...blyplatform/ALL_ARGS_SWAP
2016-01-14 17:16:52,529 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 62EA027B082E4244B7055B0990C97447 on vertex vertex_1452791812368_0001_1_00 [A9D84B543AA74502BCE2039A7E3D8AA9]
2016-01-14 17:16:52,531 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:16:52,531 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452791812368_0001_1_01 [BDD34027789442F69585C8BC8CFE6765]
2016-01-14 17:16:52,531 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452791812368_0001_1_01 [BDD34027789442F69585C8BC8CFE6765]
2016-01-14 17:16:52,531 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452791812368_0001_1_01 [BDD34027789442F69585C8BC8CFE6765]
2016-01-14 17:16:52,531 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=2F6E339E6BD946F5AB1089D1A8150788, vertexId=vertex_1452791812368_0001_1_01 [BDD34027789442F69585C8BC8CFE6765], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:16:52,532 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=2F6E339E6BD946F5AB1089D1A8150788, vertexId=vertex_1452791812368_0001_1_01 [BDD34027789442F69585C8BC8CFE6765]
2016-01-14 17:16:52,532 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:16:52,533 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 62EA027B082E4244B7055B0990C97447 asking for 1 tasks. Headroom: 733 Task Resource: 1024 waves: 1.7
2016-01-14 17:16:52,533 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:16:52,533 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=2F6E339E6BD946F5AB1089D1A8150788, vertexName=BDD34027789442F69585C8BC8CFE6765, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:16:52,534 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=2F6E339E6BD946F5AB1089D1A8150788, vertexId=vertex_1452791812368_0001_1_01 [BDD34027789442F69585C8BC8CFE6765]
2016-01-14 17:16:52,534 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:16:52,535 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:52,535 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791812368_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BDD34027789442F69585C8BC8CFE6765, vertexId=vertex_1452791812368_0001_1_01, initRequestedTime=1452791812528, initedTime=1452791812535, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:16:52,535 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791812368_0001_1_01 [BDD34027789442F69585C8BC8CFE6765] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:16:52,536 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 17:16:52,536 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 62EA027B082E4244B7055B0990C97447 on vertex vertex_1452791812368_0001_1_00 [A9D84B543AA74502BCE2039A7E3D8AA9]
2016-01-14 17:16:52,537 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791812368_0001_1_00 [A9D84B543AA74502BCE2039A7E3D8AA9] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:52,537 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {62EA027B082E4244B7055B0990C97447=forAllWorkUnits=true, update=[1]}
2016-01-14 17:16:52,537 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452791812368_0001_1_00 [A9D84B543AA74502BCE2039A7E3D8AA9] parallelism set to 1
2016-01-14 17:16:52,537 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452791812368_0001_1_00 [A9D84B543AA74502BCE2039A7E3D8AA9] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:16:52,538 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791812368_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791812368_0001_1_00, eventCount=1
2016-01-14 17:16:52,538 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791812368_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A9D84B543AA74502BCE2039A7E3D8AA9, vertexId=vertex_1452791812368_0001_1_00, initRequestedTime=1452791812527, initedTime=1452791812538, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:16:52,538 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:16:52,538 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791812368_0001_1_00 [A9D84B543AA74502BCE2039A7E3D8AA9] with distanceFromRoot: 0
2016-01-14 17:16:52,538 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791812368_0001_1_00 [A9D84B543AA74502BCE2039A7E3D8AA9] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:16:52,539 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in A9D84B543AA74502BCE2039A7E3D8AA9
2016-01-14 17:16:52,539 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791812368_0001_1_00 [A9D84B543AA74502BCE2039A7E3D8AA9]
2016-01-14 17:16:52,539 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:52,539 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791812368_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791812368_0001_1_00, startRequestedTime=1452791812539, startedTime=1452791812539
2016-01-14 17:16:52,539 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791812368_0001_1_00 [A9D84B543AA74502BCE2039A7E3D8AA9] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:52,540 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:52,540 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791812368_0001_1][Event:TASK_STARTED]: vertexName=A9D84B543AA74502BCE2039A7E3D8AA9, taskId=task_1452791812368_0001_1_00_000000, scheduledTime=1452791812539, launchTime=1452791812539
2016-01-14 17:16:52,540 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791812368_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:52,541 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452791812368_0001_1_00 for vertex: vertex_1452791812368_0001_1_01 [BDD34027789442F69585C8BC8CFE6765] numStartedSources: 1 numSources: 1
2016-01-14 17:16:52,541 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452791812368_0001_1_01 [BDD34027789442F69585C8BC8CFE6765] with distanceFromRoot: 1
2016-01-14 17:16:52,542 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791812368_0001_1_00 [A9D84B543AA74502BCE2039A7E3D8AA9]
2016-01-14 17:16:52,542 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: A9D84B543AA74502BCE2039A7E3D8AA9 in vertex: BDD34027789442F69585C8BC8CFE6765
2016-01-14 17:16:52,542 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: BDD34027789442F69585C8BC8CFE6765 with 1 source tasks and 1 pending tasks
2016-01-14 17:16:52,543 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791812368_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452791812368_0001_1_01, startRequestedTime=1452791812542, startedTime=1452791812542
2016-01-14 17:16:52,543 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:16:52,543 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791812368_0001_1_01 [BDD34027789442F69585C8BC8CFE6765] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:16:52,544 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [54315A4EF3A34510B19F1F2E4BB9ECF2/E32AD6A461244110B633456F51F0A6D3] every/(1/1) ...blyplatform/ALL_ARGS_SWAP, VertexName: A9D84B543AA74502BCE2039A7E3D8AA9, VertexParallelism: 1, TaskAttemptID:attempt_1452791812368_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=62EA027B082E4244B7055B0990C97447, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=BDD34027789442F69585C8BC8CFE6765, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:52,544 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791812368_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:52,544 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452791812368_0001, with dag id: dag_1452791812368_0001_1
2016-01-14 17:16:52,544 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:52,545 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:16:52,545 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:16:52,546 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:16:52,546 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791812368_0001_00_000001
2016-01-14 17:16:52,546 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791812368_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:52,548 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791812368_0001_00_000001
2016-01-14 17:16:52,548 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:52,549 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791812368_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791812368_0001_00_000001, launchTime=1452791812549
2016-01-14 17:16:52,549 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:52,549 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791812368_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:52,550 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791812368_0001_00_000001
2016-01-14 17:16:52,550 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791812368_0001_1_00_000000_0] to container: [container_1452791812368_0001_00_000001]
2016-01-14 17:16:52,551 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791812368_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:52,551 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791812368_0001_00_000001 given task: attempt_1452791812368_0001_1_00_000000_0
2016-01-14 17:16:52,551 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791812368_0001_1_00_000000_0] started. Is using containerId: [container_1452791812368_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:16:52,551 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:52,551 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791812368_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791812368_0001_1_00_000000_0
2016-01-14 17:16:52,551 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791812368_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A9D84B543AA74502BCE2039A7E3D8AA9, taskAttemptId=attempt_1452791812368_0001_1_00_000000_0, startTime=1452791812551, containerId=container_1452791812368_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791812368_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:16:52,552 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791812368_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:52,552 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791812368_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:52 Starting to run new task attempt: attempt_1452791812368_0001_1_00_000000_0
2016-01-14 17:16:52,552 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791812368_0001_1_00_000000_0
2016-01-14 17:16:52,552 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:52,552 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:52,552 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:52,552 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:52,553 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:52,553 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:52,553 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [54315A4EF3A34510B19F1F2E4BB9ECF2/E32AD6A461244110B633456F51F0A6D3] every/(1/1) ...blyplatform/ALL_ARGS_SWAP, VertexName: A9D84B543AA74502BCE2039A7E3D8AA9, VertexParallelism: 1, TaskAttemptID:attempt_1452791812368_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=62EA027B082E4244B7055B0990C97447, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=BDD34027789442F69585C8BC8CFE6765, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:16:52,553 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=769130496
2016-01-14 17:16:52,553 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791812368_0001_1_00_000000_0
2016-01-14 17:16:52,553 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:52,553 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:52,554 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791812368_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:16:52,554 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=62EA027B082E4244B7055B0990C97447, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:16:52,554 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=BDD34027789442F69585C8BC8CFE6765, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:16:52,554 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:52,555 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:52,555 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: BDD34027789442F69585C8BC8CFE6765
2016-01-14 17:16:52,556 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:16:52,556 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: BDD34027789442F69585C8BC8CFE6765
2016-01-14 17:16:52,556 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: BDD34027789442F69585C8BC8CFE6765
2016-01-14 17:16:52,557 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:52,560 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A9D84B543AA74502BCE2039A7E3D8AA9, ordinal: 0
2016-01-14 17:16:52,560 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A9D84B543AA74502BCE2039A7E3D8AA9, mem on start (mb), free: 605, total: 733, max: 733
2016-01-14 17:16:52,560 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:52,560 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:52,561 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:52,554 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:52,561 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 62EA027B082E4244B7055B0990C97447
2016-01-14 17:16:52,561 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 62EA027B082E4244B7055B0990C97447
2016-01-14 17:16:52,562 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:16:52,563 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 62EA027B082E4244B7055B0990C97447
2016-01-14 17:16:52,563 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 62EA027B082E4244B7055B0990C97447
2016-01-14 17:16:52,563 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:52,563 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:52,563 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:16:52,563 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:16:52,563 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:52,563 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:52,563 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 769130496, TotalAvailable: 515317432, TotalRequested/TotalJVMHeap:0.14
2016-01-14 17:16:52,564 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, BDD34027789442F69585C8BC8CFE6765, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:16:52,564 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 62EA027B082E4244B7055B0990C97447, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:16:52,564 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:52,564 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 62EA027B082E4244B7055B0990C97447 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:52,564 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:52,564 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:52,564 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 62EA027B082E4244B7055B0990C97447
2016-01-14 17:16:52,564 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:52,564 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 62EA027B082E4244B7055B0990C97447
2016-01-14 17:16:52,564 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:52,565 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791812368_0001_1_00_000000_0
2016-01-14 17:16:52,566 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:16:52,566 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@523e5303
2016-01-14 17:16:52,567 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@523e5303
2016-01-14 17:16:52,567 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:16:52,567 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:16:52,567 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:52,568 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 62EA027B082E4244B7055B0990C97447
2016-01-14 17:16:52,568 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 72DCA9CA57514A9C9CE58BE70F2FB197
2016-01-14 17:16:52,568 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 72DCA9CA57514A9C9CE58BE70F2FB197
2016-01-14 17:16:52,569 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@2b8f32a3; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@464581ac; comparator=cascading.tuple.hadoop.util.TupleComparator@7d44673e; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:16:52,569 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:16:52,569 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:16:52,569 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:52,586 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:16:52,586 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:16:52,586 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:16:52,587 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:16:52,587 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:16:52,587 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 17:16:52,587 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:16:52,587 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A9D84B543AA74502BCE2039A7E3D8AA9, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:16:52,589 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452791812368_0001_1_00_000000_0
2016-01-14 17:16:52,589 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A9D84B543AA74502BCE2039A7E3D8AA9, mem on close (mb), free: 502, total: 733, max: 733
2016-01-14 17:16:52,590 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:16:52,590 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:16:52,590 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 17:16:52,590 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:16:52,592 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:16:52 Completed running task attempt: attempt_1452791812368_0001_1_00_000000_0
2016-01-14 17:16:52,599 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452791812368_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:16:52,599 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=769130496, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452791812588, Process_Duration=1, Process_End_Time=1452791812589, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:16:52,599 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791812368_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452791812368_0001_1_00, eventCount=1
2016-01-14 17:16:52,599 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 17:16:52,599 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:52,600 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791812368_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A9D84B543AA74502BCE2039A7E3D8AA9, taskAttemptId=attempt_1452791812368_0001_1_00_000000_0, startTime=1452791812551, finishTime=1452791812600, timeTaken=49, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=769130496, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791812588, Process_Duration=1, Process_End_Time=1452791812589, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:52,600 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791812368_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:16:52,600 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452791812368_0001_1_00_000000_0
2016-01-14 17:16:52,600 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:52,600 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791812368_0001_00_000001
2016-01-14 17:16:52,600 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791812368_0001_1][Event:TASK_FINISHED]: vertexName=A9D84B543AA74502BCE2039A7E3D8AA9, taskId=task_1452791812368_0001_1_00_000000, startTime=1452791812551, finishTime=1452791812600, timeTaken=49, status=SUCCEEDED, successfulAttemptID=attempt_1452791812368_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=769130496, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791812588, Process_Duration=1, Process_End_Time=1452791812589, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:52,601 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791812368_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:16:52,601 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791812368_0001_1_00 [A9D84B543AA74502BCE2039A7E3D8AA9] : 1
2016-01-14 17:16:52,601 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:52,601 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452791812368_0001_1_00 [A9D84B543AA74502BCE2039A7E3D8AA9]
2016-01-14 17:16:52,601 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:52,602 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791812368_0001_1][Event:VERTEX_FINISHED]: vertexName=A9D84B543AA74502BCE2039A7E3D8AA9, vertexId=vertex_1452791812368_0001_1_00, initRequestedTime=1452791812527, initedTime=1452791812538, startRequestedTime=1452791812539, startedTime=1452791812539, finishTime=1452791812601, timeTaken=62, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=769130496, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791812588, Process_Duration=1, Process_End_Time=1452791812589, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452791812551, firstTasksToStart=[ task_1452791812368_0001_1_00_000000 ], lastTaskFinishTime=1452791812600, lastTasksToFinish=[ task_1452791812368_0001_1_00_000000 ], minTaskDuration=49, maxTaskDuration=49, avgTaskDuration=49.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452791812368_0001_1_00_000000 ], longestDurationTasks=[ task_1452791812368_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:16:52,602 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791812368_0001_1_00 [A9D84B543AA74502BCE2039A7E3D8AA9] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:16:52,602 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:52,602 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791812368_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:16:52,602 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791812368_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791812368_0001_00_000001, stoppedTime=1452791812602, exitStatus=0
2016-01-14 17:16:52,602 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791812368_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:52,602 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:52,602 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452791812368_0001_1_01 [BDD34027789442F69585C8BC8CFE6765] attempt: attempt_1452791812368_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:16:52,602 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: BDD34027789442F69585C8BC8CFE6765
2016-01-14 17:16:52,603 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452791812368_0001_1_01 [BDD34027789442F69585C8BC8CFE6765]
2016-01-14 17:16:52,603 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791812368_0001_1_00 [A9D84B543AA74502BCE2039A7E3D8AA9] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:16:52,603 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791812368_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:16:52,603 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791812368_0001_1][Event:TASK_STARTED]: vertexName=BDD34027789442F69585C8BC8CFE6765, taskId=task_1452791812368_0001_1_01_000000, scheduledTime=1452791812603, launchTime=1452791812603
2016-01-14 17:16:52,603 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:16:52,603 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791812368_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:16:52,603 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791812368_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:52,604 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452791812368_0001_1_01 [BDD34027789442F69585C8BC8CFE6765]
2016-01-14 17:16:52,604 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [54315A4EF3A34510B19F1F2E4BB9ECF2/E32AD6A461244110B633456F51F0A6D3] every/(1/1) ...blyplatform/ALL_ARGS_SWAP, VertexName: BDD34027789442F69585C8BC8CFE6765, VertexParallelism: 1, TaskAttemptID:attempt_1452791812368_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A9D84B543AA74502BCE2039A7E3D8AA9, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=2F6E339E6BD946F5AB1089D1A8150788, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:52,604 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791812368_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:16:52,604 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:16:52,605 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452791812368_0001_00_000002
2016-01-14 17:16:52,605 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791812368_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:16:52,606 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452791812368_0001_00_000002
2016-01-14 17:16:52,606 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:16:52,606 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791812368_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:16:52,607 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:16:52,607 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791812368_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452791812368_0001_00_000002, launchTime=1452791812606
2016-01-14 17:16:52,607 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452791812368_0001_00_000002
2016-01-14 17:16:52,608 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452791812368_0001_1_01_000000_0] to container: [container_1452791812368_0001_00_000002]
2016-01-14 17:16:52,608 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791812368_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:16:52,608 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452791812368_0001_1_01_000000_0] started. Is using containerId: [container_1452791812368_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:16:52,608 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791812368_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BDD34027789442F69585C8BC8CFE6765, taskAttemptId=attempt_1452791812368_0001_1_01_000000_0, startTime=1452791812608, containerId=container_1452791812368_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452791812368_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:16:52,608 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791812368_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:16:52,608 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:16:52,608 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791812368_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:16:52,608 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452791812368_0001_00_000002 given task: attempt_1452791812368_0001_1_01_000000_0
2016-01-14 17:16:52,609 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791812368_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452791812368_0001_1_01_000000_0
2016-01-14 17:16:52 Starting to run new task attempt: attempt_1452791812368_0001_1_01_000000_0
2016-01-14 17:16:52,609 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452791812368_0001_1_01_000000_0
2016-01-14 17:16:52,609 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:52,609 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:16:52,609 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:16:52,610 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:16:52,610 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:16:52,610 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:16:52,610 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [54315A4EF3A34510B19F1F2E4BB9ECF2/E32AD6A461244110B633456F51F0A6D3] every/(1/1) ...blyplatform/ALL_ARGS_SWAP, VertexName: BDD34027789442F69585C8BC8CFE6765, VertexParallelism: 1, TaskAttemptID:attempt_1452791812368_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A9D84B543AA74502BCE2039A7E3D8AA9, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=2F6E339E6BD946F5AB1089D1A8150788, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:16:52,610 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=769130496
2016-01-14 17:16:52,611 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452791812368_0001_1_01_000000_0
2016-01-14 17:16:52,611 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:16:52,611 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:52,611 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452791812368_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:16:52,611 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452791812368_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:16:52,611 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452791812368_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:16:52,611 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452791812368_0001_00_000001, Exiting
2016-01-14 17:16:52,611 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791812368_0001_00_000001
2016-01-14 17:16:52,612 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=A9D84B543AA74502BCE2039A7E3D8AA9, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:16:52,612 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791812368_0001_00_000001
2016-01-14 17:16:52,612 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:16:52,612 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: A9D84B543AA74502BCE2039A7E3D8AA9
2016-01-14 17:16:52,612 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:52,613 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=2F6E339E6BD946F5AB1089D1A8150788, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:16:52,613 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:16:52,613 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 2F6E339E6BD946F5AB1089D1A8150788
2016-01-14 17:16:52,613 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:16:52,614 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 2F6E339E6BD946F5AB1089D1A8150788
2016-01-14 17:16:52,615 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:16:52,617 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791812368_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:16:52,617 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452791812368_0001_00_000001 completed successfully
2016-01-14 17:16:52,617 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791812368_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:52,618 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BDD34027789442F69585C8BC8CFE6765, ordinal: 1
2016-01-14 17:16:52,619 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BDD34027789442F69585C8BC8CFE6765, mem on start (mb), free: 495, total: 733, max: 733
2016-01-14 17:16:52,619 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:16:52,619 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:16:52,619 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 692217408, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:16:52,619 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:16:52,619 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 692217408, type: INPUT, componentVertexName: A9D84B543AA74502BCE2039A7E3D8AA9
2016-01-14 17:16:52,620 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: A9D84B543AA74502BCE2039A7E3D8AA9
2016-01-14 17:16:52,620 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:16:52,624 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:16:52,624 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 2F6E339E6BD946F5AB1089D1A8150788
2016-01-14 17:16:52,625 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:16:52,625 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:16:52,625 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:16:52,625 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:16:52,625 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:16:52,625 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:16:52,625 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 692217408, TotalRequestedScaled: 6.389699150769231E8, TotalJVMHeap: 769130496, TotalAvailable: 515317432, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:16:52,625 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 2F6E339E6BD946F5AB1089D1A8150788, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:16:52,625 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, A9D84B543AA74502BCE2039A7E3D8AA9, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=692217408, allocated=515317432
2016-01-14 17:16:52,625 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:16:52,625 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: A9D84B543AA74502BCE2039A7E3D8AA9 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:16:52,626 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:16:52,626 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:16:52,626 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: A9D84B543AA74502BCE2039A7E3D8AA9
2016-01-14 17:16:52,627 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:16:52,627 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:16:52,628 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: A9D84B543AA74502BCE2039A7E3D8AA9 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:16:52,628 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=692217408, postMergeMem=0, RuntimeTotalAvailable=515317432. Updated to: ShuffleMem=515317432, postMergeMem: 0
2016-01-14 17:16:52,628 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=515317432, maxSingleShuffleLimit=128829360, mergeThreshold=463785696, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:16:52,628 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:16:52,629 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: A9D84B543AA74502BCE2039A7E3D8AA9
2016-01-14 17:16:52,629 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:16:52,629 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452791812368_0001_1_01_000000_0
2016-01-14 17:16:52,630 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452791812368_0001_1_00_000000_0_10339, runDuration: 42906, ]
2016-01-14 17:16:52,631 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452791812368_0001_1_00_000000_0_10339] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452791812631, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 17:16:52,631 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : A9D84B543AA74502BCE2039A7E3D8AA9
2016-01-14 17:16:52,631 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 17:16:52,632 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:16:52,632 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [A9D84B543AA74502BCE2039A7E3D8AA9] #57 in 2ms
2016-01-14 17:16:52,632 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 72DCA9CA57514A9C9CE58BE70F2FB197
2016-01-14 17:16:52,632 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_ARGS_SWAP"], id: 2F6E339E6BD946F5AB1089D1A8150788
2016-01-14 17:16:52,632 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_ARGS_SWAP"]
2016-01-14 17:16:52,633 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 72DCA9CA57514A9C9CE58BE70F2FB197, for 1 inputs
2016-01-14 17:16:52,633 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [A9D84B543AA74502BCE2039A7E3D8AA9] #57, status:false, isInterrupted:false
2016-01-14 17:16:52,633 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:16:52,634 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 17:16:52,634 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:52,634 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:16:52,634 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 17:16:52,634 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BDD34027789442F69585C8BC8CFE6765, all 1 inputs ready in: 00:00:00.001
2016-01-14 17:16:52,634 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : A9D84B543AA74502BCE2039A7E3D8AA9
2016-01-14 17:16:52,635 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@117d1979; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@2c135fe7
2016-01-14 17:16:52,635 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:52,635 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(92)) - caught throwable
cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:52,635 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:16:52,636 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:16:52,636 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452791812368_0001_1_01_000000_0
2016-01-14 17:16:52,636 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452791812368_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:16:52,636 INFO  task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1452791812368_0001_1_01_000000_0
cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-01-14 17:16:52,638 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791812368_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BDD34027789442F69585C8BC8CFE6765, taskAttemptId=attempt_1452791812368_0001_1_01_000000_0, startTime=1452791812608, finishTime=1452791812638, timeTaken=30, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=769130496, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791812634, Process_Duration=2, Process_End_Time=1452791812636, Read_Duration=0, Tuples_Read=1
2016-01-14 17:16:52,638 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791812368_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-01-14 17:16:52,638 INFO  impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1452791812368_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-01-14 17:16:52,638 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791812368_0001_1][Event:TASK_FINISHED]: vertexName=BDD34027789442F69585C8BC8CFE6765, taskId=task_1452791812368_0001_1_01_000000, startTime=1452791812608, finishTime=1452791812638, timeTaken=30, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
], counters=Counters: 0
2016-01-14 17:16:52,638 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452791812368_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-01-14 17:16:52,638 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:16:52,638 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452791812368_0001_1_01 [BDD34027789442F69585C8BC8CFE6765] : 1
2016-01-14 17:16:52,638 INFO  impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1452791812368_0001_1_01 [BDD34027789442F69585C8BC8CFE6765] because task failed: task_1452791812368_0001_1_01_000000
2016-01-14 17:16:52,639 INFO  impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1452791812368_0001_1_01 [BDD34027789442F69585C8BC8CFE6765] due to trigger: OWN_TASK_FAILURE
2016-01-14 17:16:52,639 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-01-14 17:16:52,639 INFO  impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1452791812368_0001_1_01 [BDD34027789442F69585C8BC8CFE6765]
2016-01-14 17:16:52,639 INFO  impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=2F6E339E6BD946F5AB1089D1A8150788, vertexId=vertex_1452791812368_0001_1_01 [BDD34027789442F69585C8BC8CFE6765]
2016-01-14 17:16:52,640 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:16:52,639 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:16:52,641 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:16:52,640 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791812368_0001_1][Event:VERTEX_FINISHED]: vertexName=BDD34027789442F69585C8BC8CFE6765, vertexId=vertex_1452791812368_0001_1_01, initRequestedTime=1452791812528, initedTime=1452791812535, startRequestedTime=1452791812542, startedTime=1452791812542, finishTime=1452791812640, timeTaken=98, status=FAILED, diagnostics=Task failed, taskId=task_1452791812368_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1452791812368_0001_1_01 [BDD34027789442F69585C8BC8CFE6765] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1452791812608, firstTasksToStart=[ task_1452791812368_0001_1_01_000000 ], lastTaskFinishTime=1452791812638, lastTasksToFinish=[ task_1452791812368_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[  ], longestDurationTasks=[  ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-01-14 17:16:52,641 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:16:52,641 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452791812368_0001_1_01 [BDD34027789442F69585C8BC8CFE6765] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-01-14 17:16:52,644 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791812368_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452791812368_0001_00_000002, stoppedTime=1452791812644, exitStatus=0
2016-01-14 17:16:52,644 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=769130496, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452791812634, Process_Duration=2, Process_End_Time=1452791812636, Read_Duration=0, Tuples_Read=1]]
2016-01-14 17:16:52,644 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BDD34027789442F69585C8BC8CFE6765, mem on close (mb), free: 488, total: 733, max: 733
2016-01-14 17:16:52,644 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=BDD34027789442F69585C8BC8CFE6765, index=1
2016-01-14 17:16:52,645 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: A9D84B543AA74502BCE2039A7E3D8AA9
2016-01-14 17:16:52,645 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:16:52,644 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791812368_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:16:52,644 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:16:52,645 INFO  node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1452791812368_0001_1_01_000000_0 failed: true container: container_1452791812368_0001_00_000002 numFailedTAs: 0
2016-01-14 17:16:52,645 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452791812368_0001_1_01 [BDD34027789442F69585C8BC8CFE6765] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-01-14 17:16:52,645 INFO  impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-01-14 17:16:52,645 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452791812368_0001_1
2016-01-14 17:16:52,645 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1452791812368_0001_1_00 [A9D84B543AA74502BCE2039A7E3D8AA9]
2016-01-14 17:16:52,645 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 2F6E339E6BD946F5AB1089D1A8150788 for vertex: vertex_1452791812368_0001_1_01 [BDD34027789442F69585C8BC8CFE6765]
2016-01-14 17:16:52,646 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452791812368_0001_1, queueSize=0
2016-01-14 17:16:52,646 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:16:52,647 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452791812368_0001_1][Event:DAG_FINISHED]: dagId=dag_1452791812368_0001_1, startTime=1452791812526, finishTime=1452791812646, timeTaken=120, status=FAILED, diagnostics=Vertex failed, vertexName=BDD34027789442F69585C8BC8CFE6765, vertexId=vertex_1452791812368_0001_1_01, diagnostics=[Task failed, taskId=task_1452791812368_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
	at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
	at cascading.TestAggregator.complete(TestAggregator.java:96)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
	at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
	at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
	at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1452791812368_0001_1_01 [BDD34027789442F69585C8BC8CFE6765] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=769130496, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452791812588, Process_Duration=1, Process_End_Time=1452791812589, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:16:52,647 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452791812368_0001_1 finished with state: FAILED
2016-01-14 17:16:52,647 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452791812368_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:16:52,647 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452791812368_0001_00_000002. Relying on regular task shutdown for it to end
2016-01-14 17:16:52 Completed Dag: dag_1452791812368_0001_1
2016-01-14 17:16:52,647 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452791812368_0001_1, dagState=FAILED
2016-01-14 17:16:52,647 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452791812368_0001_1_post
2016-01-14 17:16:52,647 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:16:52,648 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:16:52,648 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791812368_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:16:52,648 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:16:52,648 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:16:52,648 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:16:52,648 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:16:52,648 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=BDD34027789442F69585C8BC8CFE6765, sourceVertex=A9D84B543AA74502BCE2039A7E3D8AA9
2016-01-14 17:16:52,649 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=BDD34027789442F69585C8BC8CFE6765, sourceVertex=2F6E339E6BD946F5AB1089D1A8150788
2016-01-14 17:16:52,649 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452791812368_0001_00_000002
2016-01-14 17:16:52,649 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452791812368_0001_00_000002
2016-01-14 17:16:52,649 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452791812368_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-01-14 17:16:52,649 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1452791812368_0001_00_000002 completed but with errors
2016-01-14 17:16:52,649 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1452791812368_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-01-14 17:16:52,649 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452791812368_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:16:52,649 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452791812368_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-01-14 17:16:52,658 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...blyplatform/ALL_ARGS_SWAP, applicationId=application_1452791812368_0001
2016-01-14 17:16:52,659 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:16:52,659 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:16:52,659 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:16:52,659 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:16:52,659 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:16:52,660 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:16:52,660 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:16:52,660 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:16:52,661 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:16:52,661 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:16:52,661 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:16:52,661 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:16:52,662 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:16:52,666 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/54315A4EF3A34510B19F1F2E4BB9ECF2/E32AD6A461244110B633456F51F0A6D3/.tez/application_1452791812368_0001
2016-01-14 17:16:52,667 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping all jobs
2016-01-14 17:16:52,667 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping: (1/1) ...blyplatform/ALL_ARGS_SWAP
2016-01-14 17:16:52,668 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopped all jobs
2016-01-14 17:16:52,668 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [every]  completed in: 00:00.317
2016-01-14 17:16:52,668 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_ARGS_SWAP

Standard error

2016-01-14 17:16:18 Running Dag: dag_1452791777512_0001_1
2016-01-14 17:16:18 Starting to run new task attempt: attempt_1452791777512_0001_1_00_000000_0
2016-01-14 17:16:19 Completed running task attempt: attempt_1452791777512_0001_1_00_000000_0
2016-01-14 17:16:19 Starting to run new task attempt: attempt_1452791777512_0001_1_01_000000_0
2016-01-14 17:16:19 Completed running task attempt: attempt_1452791777512_0001_1_01_000000_0
2016-01-14 17:16:19 Completed Dag: dag_1452791777512_0001_1
2016-01-14 17:16:20 Running Dag: dag_1452791780042_0001_1
2016-01-14 17:16:20 Starting to run new task attempt: attempt_1452791780042_0001_1_00_000000_0
2016-01-14 17:16:20 Completed running task attempt: attempt_1452791780042_0001_1_00_000000_0
2016-01-14 17:16:20 Starting to run new task attempt: attempt_1452791780042_0001_1_01_000000_0
2016-01-14 17:16:20 Completed running task attempt: attempt_1452791780042_0001_1_01_000000_0
2016-01-14 17:16:20 Completed Dag: dag_1452791780042_0001_1
2016-01-14 17:16:21 Running Dag: dag_1452791781042_0001_1
2016-01-14 17:16:21 Starting to run new task attempt: attempt_1452791781042_0001_1_00_000000_0
2016-01-14 17:16:21 Completed running task attempt: attempt_1452791781042_0001_1_00_000000_0
2016-01-14 17:16:21 Starting to run new task attempt: attempt_1452791781042_0001_1_01_000000_0
2016-01-14 17:16:21 Completed running task attempt: attempt_1452791781042_0001_1_01_000000_0
2016-01-14 17:16:21 Completed Dag: dag_1452791781042_0001_1
2016-01-14 17:16:22 Running Dag: dag_1452791781907_0001_1
2016-01-14 17:16:22 Starting to run new task attempt: attempt_1452791781907_0001_1_00_000000_0
2016-01-14 17:16:22 Completed running task attempt: attempt_1452791781907_0001_1_00_000000_0
2016-01-14 17:16:22 Starting to run new task attempt: attempt_1452791781907_0001_1_01_000000_0
2016-01-14 17:16:22 Completed running task attempt: attempt_1452791781907_0001_1_01_000000_0
2016-01-14 17:16:22 Completed Dag: dag_1452791781907_0001_1
2016-01-14 17:16:22 Running Dag: dag_1452791782723_0001_1
2016-01-14 17:16:22 Starting to run new task attempt: attempt_1452791782723_0001_1_00_000000_0
2016-01-14 17:16:23 Completed running task attempt: attempt_1452791782723_0001_1_00_000000_0
2016-01-14 17:16:23 Starting to run new task attempt: attempt_1452791782723_0001_1_01_000000_0
2016-01-14 17:16:23 Completed running task attempt: attempt_1452791782723_0001_1_01_000000_0
2016-01-14 17:16:23 Completed Dag: dag_1452791782723_0001_1
2016-01-14 17:16:23 Running Dag: dag_1452791783520_0001_1
2016-01-14 17:16:23 Starting to run new task attempt: attempt_1452791783520_0001_1_00_000000_0
2016-01-14 17:16:23 Completed running task attempt: attempt_1452791783520_0001_1_00_000000_0
2016-01-14 17:16:23 Starting to run new task attempt: attempt_1452791783520_0001_1_01_000000_0
2016-01-14 17:16:23 Completed running task attempt: attempt_1452791783520_0001_1_01_000000_0
2016-01-14 17:16:23 Completed Dag: dag_1452791783520_0001_1
2016-01-14 17:16:24 Running Dag: dag_1452791784281_0001_1
2016-01-14 17:16:24 Starting to run new task attempt: attempt_1452791784281_0001_1_00_000000_0
2016-01-14 17:16:24 Completed running task attempt: attempt_1452791784281_0001_1_00_000000_0
2016-01-14 17:16:24 Starting to run new task attempt: attempt_1452791784281_0001_1_01_000000_0
2016-01-14 17:16:24 Completed running task attempt: attempt_1452791784281_0001_1_01_000000_0
2016-01-14 17:16:24 Completed Dag: dag_1452791784281_0001_1
2016-01-14 17:16:25 Running Dag: dag_1452791784898_0001_1
2016-01-14 17:16:25 Starting to run new task attempt: attempt_1452791784898_0001_1_00_000000_0
2016-01-14 17:16:25 Completed running task attempt: attempt_1452791784898_0001_1_00_000000_0
2016-01-14 17:16:25 Starting to run new task attempt: attempt_1452791784898_0001_1_01_000000_0
2016-01-14 17:16:25 Completed running task attempt: attempt_1452791784898_0001_1_01_000000_0
2016-01-14 17:16:25 Completed Dag: dag_1452791784898_0001_1
2016-01-14 17:16:25 Running Dag: dag_1452791785566_0001_1
2016-01-14 17:16:25 Starting to run new task attempt: attempt_1452791785566_0001_1_00_000000_0
2016-01-14 17:16:25 Completed running task attempt: attempt_1452791785566_0001_1_00_000000_0
2016-01-14 17:16:25 Starting to run new task attempt: attempt_1452791785566_0001_1_01_000000_0
2016-01-14 17:16:25 Completed Dag: dag_1452791785566_0001_1
2016-01-14 17:16:26 Running Dag: dag_1452791786186_0001_1
2016-01-14 17:16:26 Starting to run new task attempt: attempt_1452791786186_0001_1_00_000000_0
2016-01-14 17:16:26 Completed running task attempt: attempt_1452791786186_0001_1_00_000000_0
2016-01-14 17:16:26 Starting to run new task attempt: attempt_1452791786186_0001_1_01_000000_0
2016-01-14 17:16:26 Completed running task attempt: attempt_1452791786186_0001_1_01_000000_0
2016-01-14 17:16:26 Completed Dag: dag_1452791786186_0001_1
2016-01-14 17:16:27 Running Dag: dag_1452791786839_0001_1
2016-01-14 17:16:27 Starting to run new task attempt: attempt_1452791786839_0001_1_00_000000_0
2016-01-14 17:16:27 Completed running task attempt: attempt_1452791786839_0001_1_00_000000_0
2016-01-14 17:16:27 Starting to run new task attempt: attempt_1452791786839_0001_1_01_000000_0
2016-01-14 17:16:27 Completed Dag: dag_1452791786839_0001_1
2016-01-14 17:16:27 Running Dag: dag_1452791787676_0001_1
2016-01-14 17:16:27 Starting to run new task attempt: attempt_1452791787676_0001_1_00_000000_0
2016-01-14 17:16:28 Completed running task attempt: attempt_1452791787676_0001_1_00_000000_0
2016-01-14 17:16:28 Starting to run new task attempt: attempt_1452791787676_0001_1_01_000000_0
2016-01-14 17:16:28 Completed Dag: dag_1452791787676_0001_1
2016-01-14 17:16:28 Running Dag: dag_1452791788418_0001_1
2016-01-14 17:16:28 Starting to run new task attempt: attempt_1452791788418_0001_1_00_000000_0
2016-01-14 17:16:28 Completed running task attempt: attempt_1452791788418_0001_1_00_000000_0
2016-01-14 17:16:28 Starting to run new task attempt: attempt_1452791788418_0001_1_01_000000_0
2016-01-14 17:16:28 Completed Dag: dag_1452791788418_0001_1
2016-01-14 17:16:29 Running Dag: dag_1452791789125_0001_1
2016-01-14 17:16:29 Starting to run new task attempt: attempt_1452791789125_0001_1_00_000000_0
2016-01-14 17:16:29 Completed running task attempt: attempt_1452791789125_0001_1_00_000000_0
2016-01-14 17:16:29 Starting to run new task attempt: attempt_1452791789125_0001_1_01_000000_0
2016-01-14 17:16:29 Completed Dag: dag_1452791789125_0001_1
2016-01-14 17:16:29 Running Dag: dag_1452791789753_0001_1
2016-01-14 17:16:29 Starting to run new task attempt: attempt_1452791789753_0001_1_00_000000_0
2016-01-14 17:16:30 Completed running task attempt: attempt_1452791789753_0001_1_00_000000_0
2016-01-14 17:16:30 Starting to run new task attempt: attempt_1452791789753_0001_1_01_000000_0
2016-01-14 17:16:30 Completed Dag: dag_1452791789753_0001_1
2016-01-14 17:16:30 Running Dag: dag_1452791790322_0001_1
2016-01-14 17:16:30 Starting to run new task attempt: attempt_1452791790322_0001_1_00_000000_0
2016-01-14 17:16:30 Completed running task attempt: attempt_1452791790322_0001_1_00_000000_0
2016-01-14 17:16:30 Starting to run new task attempt: attempt_1452791790322_0001_1_01_000000_0
2016-01-14 17:16:30 Completed running task attempt: attempt_1452791790322_0001_1_01_000000_0
2016-01-14 17:16:30 Completed Dag: dag_1452791790322_0001_1
2016-01-14 17:16:31 Running Dag: dag_1452791790961_0001_1
2016-01-14 17:16:31 Starting to run new task attempt: attempt_1452791790961_0001_1_00_000000_0
2016-01-14 17:16:31 Completed running task attempt: attempt_1452791790961_0001_1_00_000000_0
2016-01-14 17:16:31 Starting to run new task attempt: attempt_1452791790961_0001_1_01_000000_0
2016-01-14 17:16:31 Completed running task attempt: attempt_1452791790961_0001_1_01_000000_0
2016-01-14 17:16:31 Completed Dag: dag_1452791790961_0001_1
2016-01-14 17:16:31 Running Dag: dag_1452791791670_0001_1
2016-01-14 17:16:31 Starting to run new task attempt: attempt_1452791791670_0001_1_00_000000_0
2016-01-14 17:16:31 Completed running task attempt: attempt_1452791791670_0001_1_00_000000_0
2016-01-14 17:16:31 Starting to run new task attempt: attempt_1452791791670_0001_1_01_000000_0
2016-01-14 17:16:31 Completed running task attempt: attempt_1452791791670_0001_1_01_000000_0
2016-01-14 17:16:31 Completed Dag: dag_1452791791670_0001_1
2016-01-14 17:16:32 Running Dag: dag_1452791792238_0001_1
2016-01-14 17:16:32 Starting to run new task attempt: attempt_1452791792238_0001_1_00_000000_0
2016-01-14 17:16:32 Completed running task attempt: attempt_1452791792238_0001_1_00_000000_0
2016-01-14 17:16:32 Starting to run new task attempt: attempt_1452791792238_0001_1_01_000000_0
2016-01-14 17:16:32 Completed running task attempt: attempt_1452791792238_0001_1_01_000000_0
2016-01-14 17:16:32 Completed Dag: dag_1452791792238_0001_1
2016-01-14 17:16:32 Running Dag: dag_1452791792805_0001_1
2016-01-14 17:16:33 Starting to run new task attempt: attempt_1452791792805_0001_1_00_000000_0
2016-01-14 17:16:33 Completed running task attempt: attempt_1452791792805_0001_1_00_000000_0
2016-01-14 17:16:33 Starting to run new task attempt: attempt_1452791792805_0001_1_01_000000_0
2016-01-14 17:16:33 Completed running task attempt: attempt_1452791792805_0001_1_01_000000_0
2016-01-14 17:16:33 Completed Dag: dag_1452791792805_0001_1
2016-01-14 17:16:33 Running Dag: dag_1452791793402_0001_1
2016-01-14 17:16:33 Starting to run new task attempt: attempt_1452791793402_0001_1_00_000000_0
2016-01-14 17:16:33 Completed running task attempt: attempt_1452791793402_0001_1_00_000000_0
2016-01-14 17:16:33 Starting to run new task attempt: attempt_1452791793402_0001_1_01_000000_0
2016-01-14 17:16:33 Completed running task attempt: attempt_1452791793402_0001_1_01_000000_0
2016-01-14 17:16:33 Completed Dag: dag_1452791793402_0001_1
2016-01-14 17:16:34 Running Dag: dag_1452791793971_0001_1
2016-01-14 17:16:34 Starting to run new task attempt: attempt_1452791793971_0001_1_00_000000_0
2016-01-14 17:16:34 Completed running task attempt: attempt_1452791793971_0001_1_00_000000_0
2016-01-14 17:16:34 Starting to run new task attempt: attempt_1452791793971_0001_1_01_000000_0
2016-01-14 17:16:34 Completed running task attempt: attempt_1452791793971_0001_1_01_000000_0
2016-01-14 17:16:34 Completed Dag: dag_1452791793971_0001_1
2016-01-14 17:16:34 Running Dag: dag_1452791794527_0001_1
2016-01-14 17:16:34 Starting to run new task attempt: attempt_1452791794527_0001_1_00_000000_0
2016-01-14 17:16:34 Completed running task attempt: attempt_1452791794527_0001_1_00_000000_0
2016-01-14 17:16:34 Starting to run new task attempt: attempt_1452791794527_0001_1_01_000000_0
2016-01-14 17:16:34 Completed running task attempt: attempt_1452791794527_0001_1_01_000000_0
2016-01-14 17:16:34 Completed Dag: dag_1452791794527_0001_1
2016-01-14 17:16:35 Running Dag: dag_1452791795148_0001_1
2016-01-14 17:16:35 Starting to run new task attempt: attempt_1452791795148_0001_1_00_000000_0
2016-01-14 17:16:35 Completed running task attempt: attempt_1452791795148_0001_1_00_000000_0
2016-01-14 17:16:35 Starting to run new task attempt: attempt_1452791795148_0001_1_01_000000_0
2016-01-14 17:16:35 Completed running task attempt: attempt_1452791795148_0001_1_01_000000_0
2016-01-14 17:16:35 Completed Dag: dag_1452791795148_0001_1
2016-01-14 17:16:35 Running Dag: dag_1452791795705_0001_1
2016-01-14 17:16:35 Starting to run new task attempt: attempt_1452791795705_0001_1_00_000000_0
2016-01-14 17:16:35 Completed running task attempt: attempt_1452791795705_0001_1_00_000000_0
2016-01-14 17:16:35 Starting to run new task attempt: attempt_1452791795705_0001_1_01_000000_0
2016-01-14 17:16:36 Completed running task attempt: attempt_1452791795705_0001_1_01_000000_0
2016-01-14 17:16:36 Completed Dag: dag_1452791795705_0001_1
2016-01-14 17:16:36 Running Dag: dag_1452791796314_0001_1
2016-01-14 17:16:36 Starting to run new task attempt: attempt_1452791796314_0001_1_00_000000_0
2016-01-14 17:16:36 Completed running task attempt: attempt_1452791796314_0001_1_00_000000_0
2016-01-14 17:16:36 Starting to run new task attempt: attempt_1452791796314_0001_1_01_000000_0
2016-01-14 17:16:36 Completed running task attempt: attempt_1452791796314_0001_1_01_000000_0
2016-01-14 17:16:36 Completed Dag: dag_1452791796314_0001_1
2016-01-14 17:16:37 Running Dag: dag_1452791796860_0001_1
2016-01-14 17:16:37 Starting to run new task attempt: attempt_1452791796860_0001_1_00_000000_0
2016-01-14 17:16:37 Completed running task attempt: attempt_1452791796860_0001_1_00_000000_0
2016-01-14 17:16:37 Starting to run new task attempt: attempt_1452791796860_0001_1_01_000000_0
2016-01-14 17:16:37 Completed running task attempt: attempt_1452791796860_0001_1_01_000000_0
2016-01-14 17:16:37 Completed Dag: dag_1452791796860_0001_1
2016-01-14 17:16:37 Running Dag: dag_1452791797408_0001_1
2016-01-14 17:16:37 Starting to run new task attempt: attempt_1452791797408_0001_1_00_000000_0
2016-01-14 17:16:37 Completed running task attempt: attempt_1452791797408_0001_1_00_000000_0
2016-01-14 17:16:37 Starting to run new task attempt: attempt_1452791797408_0001_1_01_000000_0
2016-01-14 17:16:37 Completed Dag: dag_1452791797408_0001_1
2016-01-14 17:16:38 Running Dag: dag_1452791797897_0001_1
2016-01-14 17:16:38 Starting to run new task attempt: attempt_1452791797897_0001_1_00_000000_0
2016-01-14 17:16:38 Completed running task attempt: attempt_1452791797897_0001_1_00_000000_0
2016-01-14 17:16:38 Starting to run new task attempt: attempt_1452791797897_0001_1_01_000000_0
2016-01-14 17:16:38 Completed running task attempt: attempt_1452791797897_0001_1_01_000000_0
2016-01-14 17:16:38 Completed Dag: dag_1452791797897_0001_1
2016-01-14 17:16:38 Running Dag: dag_1452791798427_0001_1
2016-01-14 17:16:38 Starting to run new task attempt: attempt_1452791798427_0001_1_00_000000_0
2016-01-14 17:16:38 Completed running task attempt: attempt_1452791798427_0001_1_00_000000_0
2016-01-14 17:16:38 Starting to run new task attempt: attempt_1452791798427_0001_1_01_000000_0
2016-01-14 17:16:38 Completed Dag: dag_1452791798427_0001_1
2016-01-14 17:16:39 Running Dag: dag_1452791798944_0001_1
2016-01-14 17:16:39 Starting to run new task attempt: attempt_1452791798944_0001_1_00_000000_0
2016-01-14 17:16:39 Completed running task attempt: attempt_1452791798944_0001_1_00_000000_0
2016-01-14 17:16:39 Starting to run new task attempt: attempt_1452791798944_0001_1_01_000000_0
2016-01-14 17:16:39 Completed Dag: dag_1452791798944_0001_1
2016-01-14 17:16:39 Running Dag: dag_1452791799407_0001_1
2016-01-14 17:16:39 Starting to run new task attempt: attempt_1452791799407_0001_1_00_000000_0
2016-01-14 17:16:39 Completed running task attempt: attempt_1452791799407_0001_1_00_000000_0
2016-01-14 17:16:39 Starting to run new task attempt: attempt_1452791799407_0001_1_01_000000_0
2016-01-14 17:16:39 Completed Dag: dag_1452791799407_0001_1
2016-01-14 17:16:40 Running Dag: dag_1452791799961_0001_1
2016-01-14 17:16:40 Starting to run new task attempt: attempt_1452791799961_0001_1_00_000000_0
2016-01-14 17:16:40 Completed running task attempt: attempt_1452791799961_0001_1_00_000000_0
2016-01-14 17:16:40 Starting to run new task attempt: attempt_1452791799961_0001_1_01_000000_0
2016-01-14 17:16:40 Completed Dag: dag_1452791799961_0001_1
2016-01-14 17:16:40 Running Dag: dag_1452791800440_0001_1
2016-01-14 17:16:40 Starting to run new task attempt: attempt_1452791800440_0001_1_00_000000_0
2016-01-14 17:16:40 Completed running task attempt: attempt_1452791800440_0001_1_00_000000_0
2016-01-14 17:16:40 Starting to run new task attempt: attempt_1452791800440_0001_1_01_000000_0
2016-01-14 17:16:40 Completed Dag: dag_1452791800440_0001_1
2016-01-14 17:16:41 Running Dag: dag_1452791800885_0001_1
2016-01-14 17:16:41 Starting to run new task attempt: attempt_1452791800885_0001_1_00_000000_0
2016-01-14 17:16:41 Completed running task attempt: attempt_1452791800885_0001_1_00_000000_0
2016-01-14 17:16:41 Starting to run new task attempt: attempt_1452791800885_0001_1_01_000000_0
2016-01-14 17:16:41 Completed running task attempt: attempt_1452791800885_0001_1_01_000000_0
2016-01-14 17:16:41 Completed Dag: dag_1452791800885_0001_1
2016-01-14 17:16:41 Running Dag: dag_1452791801456_0001_1
2016-01-14 17:16:41 Starting to run new task attempt: attempt_1452791801456_0001_1_00_000000_0
2016-01-14 17:16:41 Completed running task attempt: attempt_1452791801456_0001_1_00_000000_0
2016-01-14 17:16:41 Starting to run new task attempt: attempt_1452791801456_0001_1_01_000000_0
2016-01-14 17:16:41 Completed running task attempt: attempt_1452791801456_0001_1_01_000000_0
2016-01-14 17:16:41 Completed Dag: dag_1452791801456_0001_1
2016-01-14 17:16:42 Running Dag: dag_1452791802029_0001_1
2016-01-14 17:16:42 Starting to run new task attempt: attempt_1452791802029_0001_1_00_000000_0
2016-01-14 17:16:42 Completed running task attempt: attempt_1452791802029_0001_1_00_000000_0
2016-01-14 17:16:42 Starting to run new task attempt: attempt_1452791802029_0001_1_01_000000_0
2016-01-14 17:16:42 Completed running task attempt: attempt_1452791802029_0001_1_01_000000_0
2016-01-14 17:16:42 Completed Dag: dag_1452791802029_0001_1
2016-01-14 17:16:42 Running Dag: dag_1452791802654_0001_1
2016-01-14 17:16:42 Starting to run new task attempt: attempt_1452791802654_0001_1_00_000000_0
2016-01-14 17:16:42 Completed running task attempt: attempt_1452791802654_0001_1_00_000000_0
2016-01-14 17:16:43 Starting to run new task attempt: attempt_1452791802654_0001_1_01_000000_0
2016-01-14 17:16:43 Completed running task attempt: attempt_1452791802654_0001_1_01_000000_0
2016-01-14 17:16:43 Completed Dag: dag_1452791802654_0001_1
2016-01-14 17:16:43 Running Dag: dag_1452791803262_0001_1
2016-01-14 17:16:43 Starting to run new task attempt: attempt_1452791803262_0001_1_00_000000_0
2016-01-14 17:16:43 Completed running task attempt: attempt_1452791803262_0001_1_00_000000_0
2016-01-14 17:16:43 Starting to run new task attempt: attempt_1452791803262_0001_1_01_000000_0
2016-01-14 17:16:43 Completed running task attempt: attempt_1452791803262_0001_1_01_000000_0
2016-01-14 17:16:43 Completed Dag: dag_1452791803262_0001_1
2016-01-14 17:16:43 Running Dag: dag_1452791803787_0001_1
2016-01-14 17:16:43 Starting to run new task attempt: attempt_1452791803787_0001_1_00_000000_0
2016-01-14 17:16:44 Completed running task attempt: attempt_1452791803787_0001_1_00_000000_0
2016-01-14 17:16:44 Starting to run new task attempt: attempt_1452791803787_0001_1_01_000000_0
2016-01-14 17:16:44 Completed running task attempt: attempt_1452791803787_0001_1_01_000000_0
2016-01-14 17:16:44 Completed Dag: dag_1452791803787_0001_1
2016-01-14 17:16:44 Running Dag: dag_1452791804288_0001_1
2016-01-14 17:16:44 Starting to run new task attempt: attempt_1452791804288_0001_1_00_000000_0
2016-01-14 17:16:44 Completed running task attempt: attempt_1452791804288_0001_1_00_000000_0
2016-01-14 17:16:44 Starting to run new task attempt: attempt_1452791804288_0001_1_01_000000_0
2016-01-14 17:16:44 Completed running task attempt: attempt_1452791804288_0001_1_01_000000_0
2016-01-14 17:16:44 Completed Dag: dag_1452791804288_0001_1
2016-01-14 17:16:44 Running Dag: dag_1452791804792_0001_1
2016-01-14 17:16:44 Starting to run new task attempt: attempt_1452791804792_0001_1_00_000000_0
2016-01-14 17:16:45 Completed running task attempt: attempt_1452791804792_0001_1_00_000000_0
2016-01-14 17:16:45 Starting to run new task attempt: attempt_1452791804792_0001_1_01_000000_0
2016-01-14 17:16:45 Completed running task attempt: attempt_1452791804792_0001_1_01_000000_0
2016-01-14 17:16:45 Completed Dag: dag_1452791804792_0001_1
2016-01-14 17:16:45 Running Dag: dag_1452791805333_0001_1
2016-01-14 17:16:45 Starting to run new task attempt: attempt_1452791805333_0001_1_00_000000_0
2016-01-14 17:16:45 Completed running task attempt: attempt_1452791805333_0001_1_00_000000_0
2016-01-14 17:16:45 Starting to run new task attempt: attempt_1452791805333_0001_1_01_000000_0
2016-01-14 17:16:45 Completed running task attempt: attempt_1452791805333_0001_1_01_000000_0
2016-01-14 17:16:45 Completed Dag: dag_1452791805333_0001_1
2016-01-14 17:16:45 Running Dag: dag_1452791805828_0001_1
2016-01-14 17:16:46 Starting to run new task attempt: attempt_1452791805828_0001_1_00_000000_0
2016-01-14 17:16:46 Completed running task attempt: attempt_1452791805828_0001_1_00_000000_0
2016-01-14 17:16:46 Starting to run new task attempt: attempt_1452791805828_0001_1_01_000000_0
2016-01-14 17:16:46 Completed running task attempt: attempt_1452791805828_0001_1_01_000000_0
2016-01-14 17:16:46 Completed Dag: dag_1452791805828_0001_1
2016-01-14 17:16:46 Running Dag: dag_1452791806424_0001_1
2016-01-14 17:16:46 Starting to run new task attempt: attempt_1452791806424_0001_1_00_000000_0
2016-01-14 17:16:46 Completed running task attempt: attempt_1452791806424_0001_1_00_000000_0
2016-01-14 17:16:46 Starting to run new task attempt: attempt_1452791806424_0001_1_01_000000_0
2016-01-14 17:16:46 Completed running task attempt: attempt_1452791806424_0001_1_01_000000_0
2016-01-14 17:16:46 Completed Dag: dag_1452791806424_0001_1
2016-01-14 17:16:47 Running Dag: dag_1452791806921_0001_1
2016-01-14 17:16:47 Starting to run new task attempt: attempt_1452791806921_0001_1_00_000000_0
2016-01-14 17:16:47 Completed running task attempt: attempt_1452791806921_0001_1_00_000000_0
2016-01-14 17:16:47 Starting to run new task attempt: attempt_1452791806921_0001_1_01_000000_0
2016-01-14 17:16:47 Completed running task attempt: attempt_1452791806921_0001_1_01_000000_0
2016-01-14 17:16:47 Completed Dag: dag_1452791806921_0001_1
2016-01-14 17:16:47 Running Dag: dag_1452791807475_0001_1
2016-01-14 17:16:47 Starting to run new task attempt: attempt_1452791807475_0001_1_00_000000_0
2016-01-14 17:16:47 Completed running task attempt: attempt_1452791807475_0001_1_00_000000_0
2016-01-14 17:16:47 Starting to run new task attempt: attempt_1452791807475_0001_1_01_000000_0
2016-01-14 17:16:47 Completed Dag: dag_1452791807475_0001_1
2016-01-14 17:16:48 Running Dag: dag_1452791807913_0001_1
2016-01-14 17:16:48 Starting to run new task attempt: attempt_1452791807913_0001_1_00_000000_0
2016-01-14 17:16:48 Completed running task attempt: attempt_1452791807913_0001_1_00_000000_0
2016-01-14 17:16:48 Starting to run new task attempt: attempt_1452791807913_0001_1_01_000000_0
2016-01-14 17:16:48 Completed running task attempt: attempt_1452791807913_0001_1_01_000000_0
2016-01-14 17:16:48 Completed Dag: dag_1452791807913_0001_1
2016-01-14 17:16:48 Running Dag: dag_1452791808590_0001_1
2016-01-14 17:16:48 Starting to run new task attempt: attempt_1452791808590_0001_1_00_000000_0
2016-01-14 17:16:48 Completed running task attempt: attempt_1452791808590_0001_1_00_000000_0
2016-01-14 17:16:48 Starting to run new task attempt: attempt_1452791808590_0001_1_01_000000_0
2016-01-14 17:16:48 Completed Dag: dag_1452791808590_0001_1
2016-01-14 17:16:49 Running Dag: dag_1452791809088_0001_1
2016-01-14 17:16:49 Starting to run new task attempt: attempt_1452791809088_0001_1_00_000000_0
2016-01-14 17:16:49 Completed running task attempt: attempt_1452791809088_0001_1_00_000000_0
2016-01-14 17:16:49 Starting to run new task attempt: attempt_1452791809088_0001_1_01_000000_0
2016-01-14 17:16:49 Completed Dag: dag_1452791809088_0001_1
2016-01-14 17:16:49 Running Dag: dag_1452791809534_0001_1
2016-01-14 17:16:49 Starting to run new task attempt: attempt_1452791809534_0001_1_00_000000_0
2016-01-14 17:16:49 Completed running task attempt: attempt_1452791809534_0001_1_00_000000_0
2016-01-14 17:16:49 Starting to run new task attempt: attempt_1452791809534_0001_1_01_000000_0
2016-01-14 17:16:49 Completed Dag: dag_1452791809534_0001_1
2016-01-14 17:16:50 Running Dag: dag_1452791809960_0001_1
2016-01-14 17:16:50 Starting to run new task attempt: attempt_1452791809960_0001_1_00_000000_0
2016-01-14 17:16:50 Completed running task attempt: attempt_1452791809960_0001_1_00_000000_0
2016-01-14 17:16:50 Starting to run new task attempt: attempt_1452791809960_0001_1_01_000000_0
2016-01-14 17:16:50 Completed Dag: dag_1452791809960_0001_1
2016-01-14 17:16:50 Running Dag: dag_1452791810416_0001_1
2016-01-14 17:16:50 Starting to run new task attempt: attempt_1452791810416_0001_1_00_000000_0
2016-01-14 17:16:50 Completed running task attempt: attempt_1452791810416_0001_1_00_000000_0
2016-01-14 17:16:50 Starting to run new task attempt: attempt_1452791810416_0001_1_01_000000_0
2016-01-14 17:16:50 Completed Dag: dag_1452791810416_0001_1
2016-01-14 17:16:51 Running Dag: dag_1452791810894_0001_1
2016-01-14 17:16:51 Starting to run new task attempt: attempt_1452791810894_0001_1_00_000000_0
2016-01-14 17:16:51 Completed running task attempt: attempt_1452791810894_0001_1_00_000000_0
2016-01-14 17:16:51 Starting to run new task attempt: attempt_1452791810894_0001_1_01_000000_0
2016-01-14 17:16:51 Completed Dag: dag_1452791810894_0001_1
2016-01-14 17:16:51 Running Dag: dag_1452791811419_0001_1
2016-01-14 17:16:51 Starting to run new task attempt: attempt_1452791811419_0001_1_00_000000_0
2016-01-14 17:16:51 Completed running task attempt: attempt_1452791811419_0001_1_00_000000_0
2016-01-14 17:16:51 Starting to run new task attempt: attempt_1452791811419_0001_1_01_000000_0
2016-01-14 17:16:51 Completed Dag: dag_1452791811419_0001_1
2016-01-14 17:16:52 Running Dag: dag_1452791811927_0001_1
2016-01-14 17:16:52 Starting to run new task attempt: attempt_1452791811927_0001_1_00_000000_0
2016-01-14 17:16:52 Completed running task attempt: attempt_1452791811927_0001_1_00_000000_0
2016-01-14 17:16:52 Starting to run new task attempt: attempt_1452791811927_0001_1_01_000000_0
2016-01-14 17:16:52 Completed Dag: dag_1452791811927_0001_1
2016-01-14 17:16:52 Running Dag: dag_1452791812368_0001_1
2016-01-14 17:16:52 Starting to run new task attempt: attempt_1452791812368_0001_1_00_000000_0
2016-01-14 17:16:52 Completed running task attempt: attempt_1452791812368_0001_1_00_000000_0
2016-01-14 17:16:52 Starting to run new task attempt: attempt_1452791812368_0001_1_01_000000_0
2016-01-14 17:16:52 Completed Dag: dag_1452791812368_0001_1